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(54) System and method for controlling object by simulating emotions and a personality in the 
object 

(57) An autonomic device behaving adaptiveiy to a 
user, comprising: (i) a recognition unit programmed to 
recognize a user-related status; pi) a personality-form- 
ing unit programmed to form personality composed of 
preselected personality elements when receiving sig- 
nals of the user-related status accumulated for a prede- 
termined period of time, based on a predetermined 
relationship between the preselected personality ele- 
ments and the user-related status; (iii) an emotion-form- 
ing unit programmed to form emotions composed of 
preselected emotion elements when receiving signals 
of the user-related status and the personality, based on 
a predetermined relationship between the preselected 
emotion elements, the user-related status, and the per- 
sonality; (iv) an autonomic behavior-establishing unit 
programmed to select a pattern of autonomic behavior 
composed of preselected elements when receiving sig- 
nals of the user-related status, the personality and the 
emotions, based on a predetermined relationship 
between patterns of behavior, the user-related status, 
the personality, and the emotions; and (v) a new behav- 
ior pattern generatonunil programmed togeneratonev^ 
patterns of behavior by selecting and combining pat- 
terns' elements randomly or under predetermined rules, 
wherein the new patterns override the previous pattern 
under predetermined conditions. 
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Description 

[0001] This invention relates to a system and method 
for controlling an object interacting with a user and envi- 
ronment, and particularly to that for adapting, to a user, s 
behavior of an object capable of behaving in an autono- 
mous manner, thereby creating behavior highly respon- 
sive to the user or environment 
[0002] Heretofore, various controlling methods have 
been available for controlling an object in accordance 10 
with a user's demands. 

[0003] In such controlling methods, normally, the user 
sets a desired target value at output and the object is 
controlled in such a way that the output matches the tar- 
get value, while feeding the output back to a control sys- is 
tern which compares the feedback and the target value 
to adjust the output. In the above, by feeding the output 
back to the system to adjust the output the output of the 
object to be controlled can approach the target value, 
thereby achieving control satisfying the user's prefer- 20 
ences. 

[0004] If a target value is inputted by the user, it is rel- 
atively easy to obtain output close to the user's prefer- 
ences. For example, a temperature of air controlled by 
an air conditioner is an easy object because the rela- 25 
tionship between output (cooling or heating air) and a 
target value (temperature) is simple, and thus anyone 
can set a thermostat of the air conditioner at a target 
value. However, if the relationship between output and a 
target value is complicated, especially in a control sys- 30 
tern wherein changing a target value affects not only 
one output but also other outputs, only a skilled person 
can change the target value. 
[0005] Further, since humans have emotions and 
such emotions affect momentary references, for exam- as 
pie, even if an object or device is designed specifically 
for one individual user, it would be difficult to control the 
object in such a way as to always satisfy the user. 
[0006] In the above conventional control system, out- 
put from the object is designed to always approach the 40 
target value. Thus, the longer the object is used, the 
easier the prediction of behavioral patterns easier for 
the user. If the user can predict the output from the 
object, the object cannot be used for creative assistance 
to the user's intellectual work or desire. 45 
[0007] An objective of the present invention is to solve 
the above problems associated with conventional con- 
trol systems, and to provide a control system which 
allows outputting an adequate value ultimately giving 
the user more satisfaction than does a value obtained so 
from the user's direct order, particularly using emotions 
or pseudo-emotions and a personality or pseudo-per- 
sonality generated in the object in response to the user 
and environment. In the above, the personality can be 
the foundation of not only the emotions but also all ss 
behaviors. Particularly, in the present invention, an 
objective is to ease preparation of numerous behavior 
patterns or combinations stored in advance, but enable 



the object to respond to any type of user qucWy, natu- 
rally, and adaptively. The object can develop with 
respect to individuality in accordance with the user or 
with use by the user. 

[0008] According to the present invention the first 
objective is solved by an autonomic device behaving 
adaptively to a user, comprising: (i) a recognition unit 
programmed to recognize a user-related status: (ii) a 
personality-forming unit programmed to form personal- 
ity composed of preselected personality elements when 
receiving signals of the user-related status accumulated 
for a predetermined period of time, based on a predeter- 
mined relationship between the preselected personality 
elements and the user-related status; (in) an emotion- 
forming unit programmed to form emotions composed 
of preselected emotion elements when receiving sig- 
nals of the user-related status and the personality, 
based on a predetermined relationship between the 
preselected emotion elements, the user-related status, 
and the personality; (iv) an autonomic behavior-estab- 
lishing unit programmed to select a pattern of auto- 
nomic behavior composed of preselected elements 
when receiving signals of the user-related status, the 
personality and the emotions, based on a predeter- 
mined relationship between patterns of behavior, the 
user-related status, the personality, and the emotions; 
and (v) a new behavior pattern generation unit pro- 
grammed to generate new patterns of behavior by 
selecting and combining patterns' elements randomly or 
under predetermined rules, wherein the new patterns 
override the previous pattern under predetermined con- 
ditions. 

[0009] In order to further enhance the possibilities, it 
is advantageous when the recognition unit further rec- 
ognizes the user's command which is not included in the 
user-related status, the device further comprises: a 
commanded behavior-establishing unit programmed to 
select a pattern of commanded behavior when receiving 
signals of the user's command; and a behavior control 
unit programmed to control behavior by combining the 
selected pattern of autonomic behavior and the 
selected pattern of commanded behavior, based on 
predetermined priority order. 
[001 0] It is also possible that there is provided a sens- 
ing unit which senses signals from the user; afirst mem- 
ory which stores and updates the recognized user- 
related status and/or the recognized command; and a 
second memory which stores and updates the formed 
personality, wherein updating takes place under prede- 
termined conditions. 

[0011] Moreover, it is advantageous when the new 
behavior pattern generation unit has a database storing 
a pool of elements of patterns of autonomous behavior. 
[0012] A further advantageous embodiment is an 
autonomous device behaving adaptively to a user, com- 
prising: (a) a sensing unit which senses signals of the 
user's command and predetermined signals relevant to 
accomplishment of the command but not part of the 
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command, said predetermined signals inducfing user- 
related signals and user-unrelated signals; (b) a recog- 
nition unit which recognizes the user's command, a cur- 
rent user-related status, and a current user-unrelated 
status based on the sensed signals of the command 5 
and the sensed predetermined signals; (c) a first mem- 
ory which stores and updates the recognized com- 
mand, the recognized user-related status, and the 
recognized user-unrelated status; (d) a pseudo-person- 
alrty-fomm'ng wit which establishes each element of w 
predetermined pseudo-personality elements based on 
the user-related status accumulated for a predeter- 
mined period of time in the first memory to form pseudo- 
personality, said pseudo-personalrty-forming unit mem- 
orizing a predetermined relationship between the pre- is 
determined pseudo-personality elements and the user- 
related status; (e) a second memory which stores and 
updates the formed pseudo-personality, wherein updat- 
ing takes place under predetermined conditions; (f) a 
pseudo-emotion-forming unit which establishes each 20 
element of predetermined pseudo-emotion elements 
based on the user-related status and the user-unrelated 
status stored in the first memory and the pseudo-per- 
sonality stored in the second memory to form pseudo- 
emotions, said pseudo-ernotion-forming unit memoriz- 25 
ing a predetermined relationship between the predeter- 
mined pseudo-emotion elements, the user-related and 
user-unrelated statuses, and the pseudo-personality; 

(g) an autonomous behavior-establishing unit which 
selects a pattern of autonomous behavior based on the 30 
user-related and user-unrelated statuses, the pseudo- 
personality, and the pseudo-emotions, said autono- 
mous behavior-forming unit memorizing a predeter- 
mined relationship between patterns of autonomous 
behavior, the user-related and user-unrelated statuses, 35 
the pseudo-personality, and the pseudo-emotions, each 
pattern constituted by plural predetermined elements; 

(h) a new behavior pattern generation unit which gener- 
ates new patterns of autonomous behavior by selecting 
and combining patterns' elements randomly or under 40 
predetermined rules, said new behavior pattern genera- 
tion unit having a database storing a pool of elements of 
patterns of autonomous behavior, said generated new 
patterns transmitted to the memory of the autonomous 
behavior-establishing unit, wherein the new patterns 45 
override the previous pattern under predetermined con- 
ditions; (i) a commanded behavior-establishing unit 
which constitutes a pattern of commanded behavior in 
accordance with the recognized user's command; (j) a 
behavior control unit which controls behavior by com- so 
Dining the selected p aUiw w cMuluwU ff MJU uuluuiu r and 
the constituted pattern of commanded behavior based 

on a predetermined priority order; and (k) an output 
device which outputs the controlled behavior. When the 
new behavior pattern generation unit is activated to gen- 55 
erate new behavior patterns (including new emotion 
expression patterns), it is not necessary to prepare 
numerous behavior patterns in advance in combination 



with the pseudo-personality, the pseudo-emotions, and 
the other information (user/external information), and 
the system is able to respond to any type of user more 
quickly and adaptive!/ 

[0013] In the above, preferably, the new behavior pat- 
tern generation unit is linked to the pseudo-personality 
forming unit, wherein the new behavior pattern genera- 
tion unit is activated at intervals correlated with the 
formed pseudo-personality. 

[0014] In the above, the external information or envi- 
ronment information can be omitted rf the device is used 
under limited circumstances or if the external or envi- 
ronment information has nothing to do with accomplish- 
ment of the user's command or if the user so desires. 
[0015] The device to be controlled in the present 
invention is not Dmited and includes any objects which 
interact with a user and which are allowed to output not 
only compulsory (commanded) behavior but also auton- 
omous behavior for the benefit of the user. For example, 
home appliances, an industrial robot, a research assist- 
ance system, or the like can be controlled by adopting 
the control system of the present invention based on the 
same principle as the above. By modifying items and 
elements included in information to be processed, the 
control device can be cotrtronedinthesameway. Inany 
Case, according to the present invention, the control 
device, to which the present invention is adapted, is 
likely to be equipped with quasMntelligence. 
[0016] Alternatively, the above-mentioned object is 
solved according to the present invention by a behavior 
control system comprising: a behavior unit programmed 
to behave based on behavior parameters; an input unit 
programmed to output signals when receiving signals 
resulting from the behavior; and a memory medium 
attachable between the behavior unit and the input unit 
and detachable therefrom, said memory medium stor- 
ing a relationship between signals inputted from the 
input unit and behavior parameters outputted tot he 
behavior unit said relationship programmed to be 
updated based on the signals from the input unit or by a 
network of a plurality of autonomic devices, each behav- 
ing adaptively to a user and comprising: a sensing unit 
which senses preselected signals from a user who uses 
the autonomic device; a behavior unit programmed to 
behave based on behavior parameters; an input unit 
programmed to output signals when receiving signals 
resulting from the behavior through the sensing unit and 
signals provided through the network; and a behavior 
parameter-forming unit provided between the behavior 
unit and the input unit, said behavior parameter-forming 
unit comprising a memory storing a relationship 
between signals inputted form the input unit and behav- 
ior parameters outputted to the behavior unit, said 
behavior parameter-forming unit programmed to update 
said relationship based on the signals from the input 
unit wherein each device is connected with each other 
devices through the network. 
[Q01 7] In the present invention, another important 
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aspect is to provide a method for adapting behavior of a 
device to user's characteristics, to which method the 
aforesaid device is adapted. 
[001 8] According to the present invention, this further 
objective is solved by a method for adapting behavior of s 
a device to a user's characteristics, comprising the 
steps of: (b) recognizing user-related status; (d) estab- 
lishing personality composed of preselected personality 
elements based on received user-related status accu- 
mulated for a predetermined period of time based on a 10 
predetermined relationship between the preselected 
personality elements and the user-related status; (f) 
establishing emotions composed of preselected emo- 
tion elements based on received user-related status 
and the personality based on a predetermined relation- is 
ship between the preselected emotion elements, the 
user-related status and the personality; (g) selecting a 
pattern of autonomic behavior composed of preselected 
elements based on received user-related status, the 
personality, and the emotions based on a predeter- 20 
mined relationship between patterns of autonomic 
behavior, the user-related status, the personality, and 
the emotions; and (h) generating new patterns of auton- 
omous behavior by selecting and combining patterns' 
elements randomly or under predetermined rules, 2s 
wherein the new patterns override the previous pattern 
under predetermined conditions. 
[0019] In order to further enhance the possibilities, it 
is advantageous when further in step (b) the user's 
command is recognized which is not included in the 30 
non-related status; further comprising the steps: Q) 
selecting a pattern of commanded behavior in accord- 
ance with the received recognized user's command; 
and (k) controlling behavior by combining the selected 
pattern of autonomic behavior and the selected pattern as 
of commanded behavior based on a predetermined pri- 
ority order. 

[0020] It is advantageous to further comprises the 
steps of (a) sensing signals from the user; (c) storing 
and updating the recognized user-related status and/or 40 
the recognized command; and (e) storing and updating 
the formed personality, wherein updating takes place 
under predetermined conditions. 
[0021] Moreover, it is possible that in step (h) a pool of 
elements of patterns of autonomous behavior is stored 45 
with the use of a database 

[0022] Other preferred embodiments of the present 
invention are laid down in further dependent claims. 
[0023] In the following, the present invention is 
explained in greater detail with respect to several so 
embodiments thereof in conjunction with the accompa- 
nying drawings, wherein: 

Figure 1 is a schematic block diagram illustrating a 
functional architecture of the control system 55 
according to the present invention; 

Figure 2 is a schematic block diagram showing an 



example of the entire system of an assist system 
according to the present invention; 

Figure 3 is a schematic flew chart showing the 
entire system of the assist system according to the 
present invention; 

Figure 4 is a schematic block diagram showing the 
user/external information recognition unit A indi- 
cated in Figure 1; 

Figure 5 is a schematic flow chart showing proc- 
esses at the pseudo-personality information-setting 
unit B indicated in Figure 1; 

Figure 6 is a schematic flow chart showing proc- 
esses at the pseudo-emotion generation unit C indi- 
cated in Figure 1; 

Figure 7 is a schematic flow chart showing proc- 
esses at the emotion expression decision unit D 
indicated in Figure 1; 

Figure 8 is a schematic flow chart showing proc- 
esses at the autonomous behavior generation unit 
E indicated in Figure 1 ; 

Figure 9 is a schematic flow chart showing proc- 
esses at the new behavior pattern generation unit G 
indicated in Figure 1; 

Figure 10 is a schematic flow chart showing proc- 
esses at the behavior planning unit H indicated in 
Figure 1; 

Figure 11 is a schematic flow chart showing proc- 
esses from generation of each item of behavior 
information based on the current behavior informa- 
tion to setting of final output behavior; 

Figure 12 is a schematic block diagram showing an 
example of the entire system of a pet robot accord- 
ing to the present invention; 

Figure 13 is a schematic flow chart showing the 
entire system of the pet robot according to the 
present invention; 

Figure 14 is a schematic block diagram showing the 
user/environment information recognition unit A 
indicated in Figure 12; 

Figure 15 is a schematic flow chart showing proc- 
esses at the pseudo-personality information-setting 
unit B indicated in Figure 12; 

Figure 16 is a schematic flow chart showing proc- 
esses at the pseudo-emotion generation unit C indi- 
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cated in Figure 12; 

Figure 17 is a schematic flow chart shewing proc- 
esses at the emotion expression decision unit D 
indicated in Figure 12; s 

Figure 18 is a schematic flow chart showing proc- 
esses at the autonomous behavior generation unit 
E indicated in Figure 12; 

10 

Figure 19 is a schematic flow chart showing proc- 
esses at the new behavior pattern generation unit Q 
indicated in Figure 12; 

Figure 20 is a schematic flow chart showing proc- is 
esses at the behavior planning unit H indicated in 
Figure 12: 



Figure 21 is a schematic flow chart showing proc- 
esses from generation of each item of behavior 20 
information based on the current behavior informa- 
tion to setting of final output behavior; 

Figure 22 is a schematic views of a display screen 
(main screen) of a touring assist system according 2s 
to the present invention; and 

Figure 23 is a schematic view of a cSsplay screen 
(second screen) of a touring assist system accord- 
ing to the present invention. 30 

[0024] The device according to the present* invention 
is an intelligent adaptive, user-friendly device. The term 
Intelligent" means that the device is able to perceive its 
surroundings, reason out its task, plan a course of 35 
action, and execute those actions. The term "adaptive" 
means both that the device is able to respond to unex- 
pected situations while it is carrying out its plans and 
also that it is able to learn from its experiences and 
improve its performance on the tasks that it commonly ao 
carries out The term "user-friendly" means not only that 
the user is able to interact in a natural manner with the 
device but also that the device comes to be viewed 
more as a "friend" or a "pet" than a "machine". In order 
to accomplish these objectives, the device has a as 
pseudo-personality and pseudo-emotions, interacts 
with Hs user on an affective level, and learns about its 
user, in addition to autonomously and spontaneously 
pursuing its own goals. 

so 

Basic Architecture 

[0025] The control system for controlling an object 
using pseudo-emotions and a pseudo-personality of the 
present invention will be explained with reference to ss 
several embodiments depicted in the figures. Figure 1 is 
a schematic block diagram illustrating a functional archi- 
tecture of the control system according to the present 



invention. 

[0026] In this figure, an autonomic device behaving 
adaptive^ to a user, comprises: a sensing unit which 
senses signals of user's command and predetermined 
user-related signals relevant to accomplishment of the 
command but not part of the command; a recognition 
unit which recognizes the user's command and a cur- 
rent user-related status based on the sensed signals of 
the command and the sensed predetermined signals; a 
first memory which stores and updates the recognized 
command and the recognized user-related status; a 
pseudo-personality-forming unit which establishes each 
element of predetermined pseudo-personality elements 
based on the user-related status accumulated for a pre- 
determined period of time in the first memory to form 
pseudo-personality, said pseudo-personality-forming 
unit having a memory storing a predetermined relation- 
ship between the predetermined pseudo-personality 
elements and the user-related status; a second memory 
which stores and updates the formed pseudo-personaJ- 
ity, wherein updating takes place under predetermined 
conditions; a pseudo-emotion-forming unit which estab- 
lishes each element of predetermined pseudo-emotion 
elements based on the user-related status stored in the 
first memory and the pseudo-personality stored in the 
second memory to form pseudo-emotions, said pseudo- 
emotion-forming unit having a memory storing a prede- 
termined relationship between the predetermined 
pseudo-emotion elements, the user-related status, and 
the pseudo-personality; an autonomic behavior-estab- 
lishing unit which selects a pattern of autonomic behav- 
ior based on the user-related status, the pseudo- 
personality, and the pseudo-emotions, said autonomic 
behavior-forming unit having a memory storing a prede- 
termined relationship between patterns of autonomic 
behavior, the user-related status, the pseudo-personal- 
ity, and the pseudo-emotions, each pattern constituted 
by plural predetermined elements; a new behavior pat- 
tern generation unit which generates new patterns of 
autonomous behavior by selecting and combining pat- 
terns' elements randomly or under predetermined rules, 
said new behavior pattern generation unit having a 
database storing a pool of elements of patterns of 
autonomous behavior, said generated new patterns 
transmitted to the memory of the autonomous behavior- 
establishing unit, wherein the new patterns override the 
previous pattern under predetermined conditions; a 
commanded behavior-establishing unit which consti- 
tutes a pattern of commanded behavior in accordance 
with the recognized user's command; a behavior control 
unit which controls behavior by combining the selected 
pattern of autonomic behavior and the constituted pat- 
tern of commanded behavior based on a predetermined 
priority order; and an output device which outputs the 
controlled behavior. 

[0027] In the above, preferably, user-unrelated infor- 
mation such as environmental information and external 
database information is included in input information in 
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order to establish oonrplex behavior in the object 
[0028] In the above, preferably, the device further 
comprises a third memory, linked to the behavior control 
unit, which stores and updates the pseudo-emotion 
formed by the pseudo-emotion-forming unit wherein s 
the second memory is further linked to the behavior 
control unit, wherein the pseudo-emotion stored in the 
third memory and the pseudo-personality stored in the 
second memory are used as parameters of the prede- 
termined priority order provided in the behavior control 10 
unit (indicated with dotted lines in Figure 1) Further, the 
first memory may be further linked to the behavior con- 
trol unit wherein the user-related status stored in the 
first memory, the pseudo-personality stored in the sec- 
ond memory, and the pseudo-emotion stored in the third is 
memory are used as parameters of the predetermined 
priority order provided in the behavior control unit In the 
above structures, the output behavior can be more 
autonomous and adaptive (indicated with a double-dot- 
ted line in Figure 1). 20 
[0029] Figures 2-1 1 show an embodiment wherein the 
present invention is adapted to a touring assist system 
which assists a user when the user works out a plan of 
touring or traveling. Figures 12-21 show an embodiment 
wherein the present invention is adapted to a pet robot 25 
which a user can keep as a pet However, the present 
invention is not limited to these embodiments, and this 
control system can be adapted to any device which is 
useful to the user as an intelligent, adaptive, user- 
friendly device. The point is that appropriate input and so 
output are selected, depending on the goal of the device 
and the user's preferences. In any case, the principle of 
the control system is the same as those illustrated in the 
figures. 

35 

Touring Assist System 

[0030] This touring assist system (hereinafter referred 
to as "assist system") is designed to assist a user when 
the user lays out a plan of a travel or tour by motorbike, 40 
car, train, or on foot The assist system rtserf has a 
pseudo-emotion which changes in response to interac- 
tion with the user or depending on the surrounding envi- 
ronments, and a pseudo-personality which grows based 
on the interaction with the user or its experience. 45 
Accordingly, the assist system assists working out the 
user's plan based on the interaction with the user and 
information on the environments (train maps, road 
maps, traffic news, etc., while the device expresses its 
pseudo-emotion or produces an autonomous response so 
or a, response in accordance with theuseTs command: 
[0031] In practice, in order to recognize the state of 
the user or the environments, this assist system com- 
prises a visual detection sensor, a touch detection sen- 
sor, an auditory detection sensor, and a means for ss 
accessing external databases, and further comprises 
an output device, such as a visual output device and an 
auditory output device, for incficating its pseudo-emo- 



tions or responses to the user. The assist system can be 
constructed so as to load a motorbike or a car therewith 
or allow the user to carry it 

Basic Structures of Touring Assist System 

[0032] Figure 2 is a schematic block diayam showing 
an example of the entire system of an assist system 
according to the present invention. As shown in this fig- 
ure, the assist system comprises: a user/external infor- 
mation recognition unit A for recognizing a state of the 
user or external information; a pseudo-personality infor- 
mation-setting unit B for setting a pseudo-personality by 
referring to the outcome of recognition by the recogni- 
tion unit A; a pseudo-emotion generation unit C for gen- 
erating pseudo-emotions based on the outcome of 
recognition by the recognition unit A and the information 
on the pseudo-personality set by the pseudo-personal- 
ity information-setting unit B; an emotion expression 
decision unit D for deciding manners of emotion expres- 
sion, such as expression methods to express laugh or 
anger on a display, based on the information on the 
pseudo-emotion; an autonomous behavior generation 
unit E for generating autonomous behavior based on 
the outcome of recognition, the information on the 
pseudo-personality, and the information on the pseudo- 
emotions; a commanded behavior generation unit F for 
generating behavior in accordance with the user's com- 
mand based on the outcome of recognition and at least 
the preceding behavior previously outputted by this sys- 
tem itself; a new behavior pattern generation unit G for 
generating new behavior patterns which are usable in 
the emotion expression decision unit D and the autono- 
mous behavior generation unit E based on the pseudo- 
personality information, the pseudo-emotion informa- 
tion, and a database storing various behavior elements 
and emotion expression elements; and a behavior plan- 
ning unit H for deciding final behavior to be outputted 
using output from each of the emotion expression deci- 
sion unit D, the autonomous behavior generation unit E, 
and the commanded behavior generated unit F, based 
on the outcome of recognition, the information on the 
pseudo-personality, and the information on the pseudo- 
emotions. As shown in Figure 3, once the system is acti- 
vated, this assist system repeats each processing step 
at units A through H until the system is turned off. The 
structure and function of each unit will be explained in 
detail below. 

User/External Information Recognition Unit 

[0033] As shown in Figure 2, the user/external infor- 
mation recognition unit A comprises various input 
devices (e.g., an auditory input device, a visual input 
device, an evaluation input device, a keyboard, a com- 
munication device, a thermometer, an flluminometer, 
and/or a watch), a user's evaluation recognition unit Al, 
a user's emotion recognition unit A2, a user's command 
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recognition unit A3, an external information recognition 
unit A4, and a current user information/characteristics 
deduction unit AS. The user's evaluation recognition 
unit A1 performs recognition-processing based on the 
information obtained via the input devices. The current s 
user information/characteristics deduction unit AS 
determines or deduces a current state of the user and 
the user's characteristics such as the user's prefer- 
ences based on the recognition outcome from the user- 
related recognition units A1 through A3. w 
[0034] In the above, the input device includes a micro- 
phone as an auditory input device, a camera as a visual 
input device, a switch or lever for inputting the user's 
evaluation as an evaluation input device, and an 
antenna, a modem, a terminal adapter, or merely an is 
input port as a communication device. However, the 
input devices are not limited to the above. Any input 
device can be used which is available on the market, 
which is capable of detecting external information visu- 
ally, aucfitorfly, and/or tactilefy, and which is capable of 20 
accessing external databases. 
[0035] The term "external information" means not only 
information on the surrounding environments, such as 
the temperature, brightness, or time, where the system 
is operated (hereinafter referred to as "environmental 2s 
information"), but also information obtained from exter- 
nal databases other than from the user or the system. 
[0036] Figure 4 is a schematic block diagram showing 
the user/external information recognition unit A in detail 
to explain processes at each of the processing units A1 30 
through AS in the user/external information recognition 
unit A. 

[0037] As shown in this figure, the user's evaluation 
recognition unit A1 establishes the user's evaluation as 
to behavior outputted by the system and sets the resutt, 35 
for example, as information on the user's evaluation, by 
calculating an evaluation value from data inputted by an 
evaluation input device composed of a lever, for exam- 
ple, which allows the user to input his/her evaluation at 
an appropriate level among plural levels, depending on 40 
the user's satisfaction. 

[0038] The user's emotion recognition unit A2 pre- 
sumes the user's current emotions based on the char- 
acteristics found in the auditory information and the 
user's facial expression, wherein an auditory analysis 45 
(analysis of frequency distributions or pitches of the 
soundA/oice) and a facial expression analysis (analysis 
based on the shapes of the eyes and the mouth) are 
conducted based on the auditory information and the 
user's facial expression inputted from the auditory input so 
device and the vfeuaf fnjxif dWcer. Further; based on 
the presumed emotions, the user's emotion recognition 
unit A2 determines the user's current emotion. In the 
above, the several patterns of emotions are obtained by 
categorizing in advance the emotion expressions of the ss 
user into several patterns of emotions such as joyful, 
sad, surprised, angry, disgusted, and fearful. The pat- 
terns of emotion expressions are recognized by subdi- 



viding each pattern into several levels. In the above, the 
presumption of the user's emotions based on the input 
information can be performed using a fuzzy logic with 
predetermined rules, or using data obtained by formu- 
lating or mapping in advance the relationship between 
emotions and characteristics of sounds/voices and 
facial expressions, or using appropriate functional equa- 
tions if regularity can be found in the relationship 
between emotions and characteristics of sounds/voices 
and facial expressions. The information inputted into the 
emotion recognition unit A2 is not limited to the above 
and includes any information related to the user's 
behavior affected by or representing the user's emo- 
tions, for example, the manner of punching the key- 
board: If the user punches the same key several times 
without a special reason when the user inputs informa- 
tion, the user is presumed to be irritated (a low level of 
anger). 

[0039] The user's command recognition unit A3 rec- 
ognizes and establishes the user's command informa- 
tion based on indirect command information obtained 
by analyzing the meaning of the sound/voice recog- 
nized by the auditory input device and by analyzing 
information on the user's gestures recognized by the 
visual input device, and based on direct command infor- 
mation inputted via the keyboard. The command infor- 
mation includes not only direct commands such as 
"search" or "lay out an action plan" but also a degree of 
requirement for the command such as "urgent" and 
"normal". The degree of requirement can be recognized 
by. for example, the loudness of the users voice or the 
speed of the user's speech inputted via the auditory 
input device, or the user's gesture inputted via the visual 
input device. The recognition means is not limited to the 
above and includes various input information such as 
the strength of punching the keyboard wherein each key 
is provided with a pressure-sensing sensor. The infor- 
mation obtained from the keyboard provided with pres- 
sure-sensing sensors can be used in the recognition 
process at the user's emotion recognition unit A2. 
[0040] The external information recognition unit A4 
establishes external information such as newly released 
information from external databases, the user's past 
record of using the system, ambient temperature, 
brightness, time, and date, by automatically collecting 
information through external information networks such 
as the Internet via the communication device and by 
processing the inputted information from various sen- 
sors such as a thermometer, illuminometer, and watch. 
[0041] The current user information/characteristics 
deduction urrif AS" dfeferrrtfnes or deduces the current 
state and the characteristics of the user based on the 
recognition results at the user-related information rec- 
ognition units Al through A3, the information on the cur- 
rent priority behavior described later, the information on 
the emotion expression patterns, and the information on 
the autonomous behavior patterns. The current user 
information/characteristics deduction unit AS then 
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updates data of the recognition information related to 
the user stored in a user information memory 1. 
[0042] In the above, the current state of the user 
includes a time period while the user is located within a 
range where the system can locate the user; the level of s 
the user's emotions; and the user's evaluation as to the 
behavior outputted by the system. The characteristics of 
the user includes frequency of the occurrence of each 
emotion, the changing rate or range of emotions, the 
types of keywords that the user uses, frequency of use io 
of keywords, and the average value of the user's evalu- 
ation as to the behavior outputted by the system. 
[0043] The output of the user's command recognition 
unit A3 is transmitted not only to the user information 
memory 1 but also to a user's command information is 
memory 2 thereby updating the information related to 
the user's command stored in the memory 2. The infor- 
mation related to the user's command includes, for 
example, the contents of the command, the object of the 
command, and the conditions of the command. 20 
[0044] The output of the external information recogni- 
tion unit A4 is transmitted to an external information 
memory 3, thereby updating the information stored in 
the memory 3. 

2S 

Pseudo-Personality Information-Setting Unit 

[0045] Figure 5 is a schematic flew chart showing 
processes at the pseudo-personal rty information-setting 
unit B. 30 
[0046] The pseudo-personality information-setting 
unit 8 is constructed so as to set, as a pseudo-person- 
ality, a group of several preliminary personalities, each 
subdividable into several levels. Based on the informa- 
tion stored in the user information memory 2, the level of 35 
each preliminary personality is set and the higher the 
level, the more the preliminary personality contributes to 
the formation of the pseudo-emotions and the action 
plan. The preliminary personalities include, for example, 
"rise and fall of emotions'', "obedience", "aggressive- 40 
ness", "curiosity", and "cheerfulness". Each preliminary 
personality is correlated with each characteristic of the 
user stored in the user information memory 1. In the 
above, "rise and fall of emotions" is correlated with the 
changing rate or range of the user's emotions, "obedi- 45 
ence" is correlated with frequency of the occurrence of 
the user's emotion "anger", "aggressiveness" is corre- 
lated with the evaluation of the behavior outputted by 
the system and frequency of the occurrence of each 
emotion in the user at the time of evaluation, "curiosity" so 
is correlated with the range of keyword* that the user 
likes to use. and "cheerfulness" is correlated with fre- 
quency of the occurrence of the user's emotion "joy" 
and positive evaluation of the behavior outputted by the 
system. Thus, in the above, if the emotions of the user ss 
intensively and often fluctuate, the level of rise and fall of 
emotions is increased. If the user gets angry often, the 
level of obedience is decreased. If the user positively 



evaluates the behavior outputted by the system and is 
delighted, the level of aggressiveness is increased. If 
the keyword that the user likes are scattered in various 
fields, the level of curiosity is increased. If the user often 
positively evaluates the behavior outputted by the sys- 
tem and is often delighted, the level of cheerfulness is 
increased. 

[0047] The pseudo-personality established at the 
pseudo-personality setting unit B is outputted to a 
pseudo-personality information memory 4 per prelimi- 
nary personality, thereby updating the information 
stored in the pseudo-personality information memory 4. 
Updating can be conducted by using the input from the 
pseudo-personality setting unit B as new pseudo-per- 
sonality information and overriding the preceding infor- 
mation, or by adding the levels of the preliminary 
personalities obtained at the pseudo-personality setting 
unit B to the prececGng information. The former method 
allows the pseudo-personality of the system to be highly 
responsive to the user's characteristics, and the latter 
method allows the pseudo-personality of the system to 
grow in accordance with frequency of use of the system 
and a lapse of time, thereby allowing the user to enjoy 
the growth of the pseudo-personality of the system. In 
particular, in the case of the latter method, if the levels 
of each preliminary personality are set at several 
detailed levels, and output from the pseudo-personality 
information-setting unit B is set to be low, it is possible 
to eqoy the growth of the pseudo-personality of the sys- 
tem which takes place over a long period of time, i.e., 
the personality can be more human-like. 

Pseudo-Emotion Generation Unit 

[0048] Figure 6 is a schematic flow chart: showing 
processes at the pseudo-emotion generation unit C. 
[0049] The pseudo-emotion generation unit C 
receives output from the user information memory 1 , the 
externa] information memory 3, and the pseudo-person- 
ality information memory 4, and. based on the above 
information, the pseudo-emotion generation unit C gen- 
erates pseudo-emotion information of the system. 
[0050] The pseudo-emotion generation unit C memo- 
rizes several predetermined preliminary emotions (such 
as "joyful", "sad", "surprised", angry", "disgusted", and 
"fearful"), and sets the level of each preliminary emotion 
based on the information inputted from the user infor- 
mation memory 1 and the external information memory 
3. In practice, the level of each preliminary emotion can 
be determined by rules or functional equations correlat- 
ing the level of each preliminary emotion with the infor- 
mation on the state of the user (such as emotion level 
and evaluation level) stored in the user information 
memory 1 and the information on the environmental 
information stored in the external information memory 
3. The relationship between the preliminary emotions, 
the state of the user, and the environmental information 
Is formulated, for example, as follows: The higher the 
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level of "anger among the user's emotions, the higher 
the level of "sadness" becomes and the lower the level 
of "joy" becomes among the user's preliminary emo- 
tions. The higher the level of evaluation as to the behav- 
ior outputted by the system, the higher the level of "joy" 5 
becomes and the lower the level of "sadness" becomes. 
The tower the ambient temperature obtained from the 
environmental information, the higher the level of "dis- 
gust" becomes. The darker the surrounding environ- 
ment, the higher the level of "fear becomes. 10 
[0051 ] The pseudo-emotion generation unit C decides 
whether a not each preliminary emotion generated at 
the unit C is outputted therefrom as pseudo-emotions 
based on a predetermined threshold set for each pre- 
liminary emotion (hereinafter referred to as "preliminary 15 
emotion threshold"). The level of a preliminary emotion 
which is not outputted is set at zero, and the level of a 
preliminary emotion which is outputted is set at an 
appropriate level based on the information on the 
pseudo-personality. A group of the above preliminary 20 
emotions are outputted as a pseudo-emotion. 
[0052] The preliminary emotion threshold is correlated 
in advance to the pseudo-personality information, and is 
determined based on the pseudo-personality informa- 
tion inputted from the pseudo-personality information 2s 
memory 4. In practice, each preliminary emotion thresh- 
old is correlated in such a way as to change in accord- 
ance with the level of each preliminary emotion in the 
pseudo-personality information. For example, the 
higher the level of "rise and fall of emotions" in the 30 
pseudo-personality, the lower the threshold of each pre- 
liminary emotion becomes. In contrast, the lower the 
level, the higher the threshold of each preliminary emo- 
tion becomes. The higher the level of "cheerfulness" in 
the pseudo-personality, the lower the threshold of only 35 
"joy" among the others and the higher the thresholds of 
the remaining preliminary emotions become. In the 
former, the higher the level of "rise and fall of emotions" 
in the pseudo-personality, the higher the possibility of 
exceeding the respective thresholds becomes, thereby ao 
increasing the possbflity of expressing the preliminary 
emotions as the pseudo-emotion, i.e., emotion fluctua- 
tion becomes remarkable. In the latter, the higher the 
level of "cheerfulness" in the pseudo-personality, the 
higher the possibility of exceeding the threshold of only 45 
"joy" becomes, thereby rendering the system joyful. In 
another example, the higher the level of "obedience" in 
the pseudo-personality, the higher the threshold of each 
preliminary emotion becomes. In contrast the lower the 
level of "obedience", the lower the threshold of each so 
preliminary emotion becomes. As descrfcecf above, by 
correlating the preliminary emotion thresholds with the 
levels of each preliminary emotion in the pseudo-per- 
sonality, the likelihood of appearance of each pseudo- 
emotion is changed depending on the pseudo-personal- 55 
ity, and as the pseudo-personality grows, the manner of 
appearance of pseudo-emotions grows. 
[0053] After calculating the threshold of each prelimi- 



nary emotion for each preliminary emotion, or during 
such calculation, each preliminary emotion threshold 
and the corresponding preliminary emotion are com- 
pared, and only preliminary emotions whose levels are 
higher than the respective preliminary emotion thresh- 
olds undergo calculation of their levels to output as a 
pseudo-emotion. The remaining preliminary emotions 
are cleared except for the set information. 
[0054] The levels to be outputted as pseudo-emotions 
are correlated with the levels of the preliminary person- 
alities in the pseudo-personality, and can be calculated 
using rules or functional equations. In practice, for 
exanple, the higher the level of "rise and fall of emo- 
tions" in the pseudo-personality, the closer to the levels 
of preliminary emotions the output levels of pseudo- 
emotions becoma In contrast, the lower the level of 
"rise and fall of emotions" in the pseudo-personality, the 
lower then the levels of preliminary emotions the output 
levels of pseudo-emotions become. 
10055] The pseudo-emotions obtained by the above 
processes are outputted to a pseudo-emotion informa- 
tion memory 5, thereby updating the data stored in the 
memory 5. 

Emotion Expression Decision Unit 

[0056] Figure 7 is a schematic flow chart showing 
processes at the emotion expression decision unit D. 
[0057] The emotion expression decision unit D judges 
the level of each preliminary emotion from the pseudo- 
emotion information obtained from the pseudo-emotion 
memory 5, and determines emotion expression to be 
outputted based on preliminary emotions whose levels 
are high or more than zero. 

[0058] The emotion expression is selected from plural 
patterns of emotion expression in relation to pseudo- 
emotions stored in advance in an emotion expression 
pattern memory 6. The emotion expression includes, in 
practice, changes in shapes of the eyes and the mouth 
or the shedding of tears if a "face" is created by a visual 
output device. Further, emotion expression can be indi- 
cated by background color of the lace" on the display, 
or by sounds or voices or conversation using an audi- 
tory output device. In the above, the emotion expression 
pattern memory 6 memorizes in advance patterns of 
each of eyes, mouth, tears, background color, sound, 
and conversation in relation to each level of each pre- 
liminary emotion. 

[Q059] The method of selecting the patterns of the 
emotion expressions at the emotion expression unit D 
can be smrply a method wherein the pattern of the emo- 
tion expression, which corresponds to the preliminary 
emotion having the highest level, is selected. However, 
preferably, a pattern of the absence of expression 
(expressionless) is created among patterns of emotion 
expressions, and if the levels of all of the preliminary 
emotions are zero, the expressionless pattern is 
selected, and if the levels of preliminary emotions, 
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except for one preliminary emotion, are zero, a pattern 
corresponding to the one preliminary emotion is 
selected. K the levels of more than one but not all pre- 
liminary emotions are more than zero, a pattern of a 
preliminary emotion having a higher level than those of 5 
the other preliminary emotions has priority over the 
other patterns. Further, among the remaining patterns, 
a pattern of a preliminary emotion having a lower level 
than the first one is selected where the first pattern does 
not apply. In the above, the first pattern and the second 10 
pattern are combined to make one combined expres- 
sion. If the levels of all of the preliminary emotions are 
more than zero, the expressionless pattern may be 
selected. Accordingly, the pseudo-emotions can be indi- 
cated in a complex manner. 15 
[0060] The emotion expression decision unit D deter- 
mines emotion expression patterns from the pseudo- 
emotions at the moment in the above-described man- 
ner, set the patterns as emotion expression behavior, 
and outputs them to the behavior planning unit H. 20 

Autonomous Behavior Generation Unit 

[0061] Figure 8 is a schematic flow chart showing 
processes at the autonomous behavior generation unit 2s 
E. 

[0062] The autonomous behavior generation unit E 
weighs plural autonomous behaviors set in an autono- 
mous behavior pattern memory 7 based on specific 
external information, the user information, the pseudo- 30 
emotion information, and the pseudo-personality infor- 
mation, in order to determine a degree of priority of 
each autonomous behavior. 

[0063] In this embodiment, the autonomous behavior 
patterns are composed of upper behavior patterns and 35 
lower behavior patterns. First, the upper behavior pat- 
terns are weighed, and after selecting an upper behav- 
ior pattern having a higher degree of importance, the 
lower behavior patterns are weighed for the selected 
upper behavior pattern, thereby selecting a lower 40 
behavior pattern having a higher decree of importance. 
[0064] The upper behavior patterns include, for exam- 
ple, "indication of newly released information", "speak- 
ing to the user", and "automatic collection of 
information". 45 
[0065] Each of the plural behavior patterns is corre- 
lated with the levels of each of the external information, 
the user information, the pseudo-emotion information, 
and the pseudo-personality, and the degree of impor- 
tance of each behavior is determined based on the level so 
of each item of information, frt practice, for axampfeV 
with regard to the external information and the user 
information, the status of updating newly released infor- 
mation, its amount the user's emotions, preferences, 
and personality, intervals of response by the user, and a ss 
time period while the user is located within a range 
where the system can locate the user are converted to 
numerical values which are then correlated with each 



degree of importance. With regard to the pseudo-emo- 
tion information and the pseudo-personality information, 
the level of each preliminary emotion and the level of 
each preliminary personality are correlated with the 
degree of importance of each behavior pattern. Correla- 
tion of each item of information with each degree of 
importance can be achieved using functional equations 
if there is regularity between them. If there is no regular- 
ity between them, data indicating their relationship can 
be obtained through experiments in advance. 
[0066] The autonomous behavior generation unit E 
assigns a degree of importance to each behavior pat- 
tern as described above, and then sets a behavior 
which has the highest degree of importance and which 
has not been selected for a given period of time since 
the same behavior was previously conducted, as auton- 
omous behavior information. In the above, the set 
behavior pattern is an upper behavior pattern, and the 
autonomous behavior generation unit E assigns 
degrees of importance to plural lower behavior patterns 
for the set upper behavior pattern. The lower behavior 
patterns include, for example, "joking", "questioning", 
"reporting the current status", and "growling", which cor- 
respond to "speaking to the user" of the upper behavior 
pattern. 

[0067] The autonomous behavior generation unit E 
assigns degrees of importance to the lower behavior 
patterns, and then sets a behavior which has the high- 
est degree of importance as autonomous behavior 
information. In an embodiment indicated in Figure 8, the 
upper behavior pattern "speaking to the user is 
selected, and further "questioning" is selected, i.a, the 
behavior "questioning" is selected. From data regarding 
the lower behavior patterns, one particular question is 
then selected, and this behavior is set as a final autono- 
mous behavior. In the above, when one data is selected 
from a group of data ranking lower (lower level or more 
detailed level) than the lower behavior patterns, the 
selection can be made at random or can be made in 
accordance with appropriate rules, e.g., if the command 
from the user can be understood from the user informa- 
tion, words in the text of the command can be used as 
keywords for a keyword search, and a question in corre- 
spondence most with the searched keyword can be 
selected. 

[0068] In this embodiment assigning degrees of 
importance to behavior patterns are conducted in two 
levels, i.e., in both the upper behavior patterns and the 
lower behavior patterns, and final autonomous behavior 
information is selected from a group of low-level data. 
The sefeclforr process fs not flmHetf to the above, and 
can be conducted in three or more levels when behavior 
patterns are categorized into three or more levels. If the 
levels of autonomous behavior patterns, to which 
degrees of importance are assigned, are categorized in 
detail, the range of autonomous behavior can be broad- 
ened. 

[0069] Further, in this embodiment, automatic collec- 
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tion of information is one of many autonomous behavior 
patterns. However, automatic collection of information 
can be independent from the other autonomous behav- 
ior patterns, and can be conducted constantly. In this 
case, intervals of collecting information and a range of s 
information only may be changed in accordance with 
the pseudo-personality information, the pseudo-emo- 
tion information, and the user information. That is, the 
above information and the levels of the automatic col- 
lection of information may be correlated with each other w 
in advance. For example, the higher the level of "aggres- 
siveness" in the pseudo-personality, or the higher the 
level of "joy" in the pseudo-emotions, or the higher the 
level of "impatient character" in the user information, the 
higher the level of the automatic collection of informa- 1S 
tion becomes, La. accordingly, intervals of collection 
becomes shorter and the range of information become 
wider. In this way, while the user is interacting with the 
system, the system can collect new information by its 
own judgement and the thus-obtained data provide 20 
higher satisfaction to the user by generation of more 
complete autonomous behavior and commanded 
behavior described below. 

New Behavior Pattern Generation Unit 2s 

[0070] Both of the emotion expression pattern infor- 
mation memory 6 and the autonomous behavior pattern 
memory 7, which are used in the emotion expression 
decision unit D and the autonomous behavior decision 30 
unit E, are updated at the new behavior pattern genera- 
tion unit Q at appropriate intervals. 
[0071] Figure 9 is a schematic flow chart showing 
processes at the new behavior pattern generation unit 
G indicated in Figure 1 . 35 
[0072] The new behavior pattern generation unit G 
judges intervals between generation of new behavior 
patterns and subsequent generation of new behavior 
patterns based on the pseudo-personality information. 
In practice, the levels of "emotion fluctuation" or 40 
"aggressiveness", for example, in the pseudo-personal- 
ity information are correlated with intervals of genera- 
tion. For example, correlation is formulated in such a 
way that the higher the level of "aggressiveness", the 
shorter the interval becomes. 45 
[0073] The new behavior pattern generation unit G 
calculates intervals of generating new behavior pat- 
terns, and then judges whether generation of new 
behavior patterns is activated based on the intervals 
and random elements. If generation is judged to be acti- so 
vated, the new behavfor pattern generation unit G com- 
bines behavior elements stored in an emotion 
expression element database 8 and an autonomous 
behavior element database 9, which memorize in 
advance a number of emotion expression patterns and ss 
autonomous behavior patterns, respectively, thereby 
setting new emotion expression patterns and new 
autonomous behavior patterns. The above combining 



processes can be conducted randomly or by using 
genetic algorithms. In practice, for example, various 
shapes of portions of a "face" such as eyes, eyebrows, 
and mouth are inputted as emotion expression ele- 
ments in the database in advance, and new expression 
patterns of "anger can be generated randomly using 
emotion expression elements stored in the database. In 
the above, the behavior patterns can include emotion 
expression patterns, i.a, simply one integrated memory 
can store these pieces of information, and processes for 
new emotion expression patterns and processes for 
new autonomous behavior patterns can be integrated. 
[0074] After generating new emotion expression pat- 
terns and new autonomous behavior patterns according 
to the above process, these new patterns are checked 
and revised with a common sense database and the 
user information to eliminate extreme or obviously dis- 
taste patterns. In practice, based on the common sense 
database including information such as "when express- 
ing 'anger*, the comer of the eye is not down", the new 
patterns are checked and revised with respect to com- 
mon sense aspects. Further, if when expression of 
"anger" is associated with "growling", the user 
expresses uncomfortable reaction, the sound associ- 
ated with "anger" is changed from loud growling" to 
"quiet growling", for exampJa Checking based on the 
user's response or evaluation is conducted using the 
user information. 

[0075] After the new behavior pattern generation unit 
G checks and revises the newly generated patterns as 
described above, the unit G temporarily memorizes the 
currently used behavior patterns corresponding to the 
newly generated patterns, and then temporarily updates 
the current behavior patterns to the new behavior pat- 
terns. These temporary new behavior patterns are eval- 
uated. 

[0076] Evaluation of the new behavior patterns is con- 
ducted mainly based on the user's evaluation in the user 
information. If evaluations of the new behavior patterns 
are not good after using the new behavior patterns for a 
given period of time, the memorized previous behavior 
patterns are resumed. If the result of evaluation is posi- 
tive, the new behavior patterns are memorized in the 
respective pattern information memoirs 6 and 7. 

Commanded Behavior Generation Unit 

[0077] The commanded behavior generation unit F 
generates commanded behavior by using information 
from the external information memory 3 in accordance 
with the user's command included in the user's com- 
mand information. In practice, in this assist system, the 
commanded behavior includes answering the user's 
questions, outlining an action plan required by the user, 
constituting a detailed action plan, and formation of sim- 
ulation of the action plan. More specifically, answering 
the user's questions means conducting a search using 
the external information memory 3 and/or the external 
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databases and displaying the search result, if the user 
commands the system to conduct a search. Outlining 
an action plan and constructing a detailed plan mean 
laying out a travel plan showing the user's destination 
and tourist attractions on the way to the destination, lay- 5 
ing out a route if the user is touring by car or motorbike, 
and laying out a train route if the user is traveling by 
train. 

[0078] In the above, commanded behavior generation 
processes at the commanded behavior generation unit 10 
F are conducted by at least referring to the previous 
behavior outputted from the system, and the information 
about the user's evaluation as to the previous behavior. 
For example, when the user commands the system to 
conduct a search, if the previously outputted behavior 15 
included a result of a search, the commanded behavior 
generation unit F determines whether the keyword used 
in the previous search was in accordance with the 
user's request based on the level of the user's evalua- 
tion as to the result of the previous search. If the level of 20 
the user's evaluation was high, the unit F judges that the 
user is satisfied with the keyword used in the search, 
and the unit F conducts a second search with the same 
keyword. If the level of the user's evaluation was low, the 
unit F judges that the user is not satisfied with the key- 2s 
word, the unit F conducts a new first search with a new 
keyword. 

[0079] As described above, upon referring to the 
user's evaluation and the previously outputted behavior, 
the next commanded behavior is generated, so that so 
behavior more suitable to the user can be generated. 
[0080] Further, the commanded behavior generation 
unit F can generate absolute behavior in addition to the 
above-mentioned commanded behavior. In practice, for 
example, risk-related keyword, such as "earthquake", as 
"frequently happening", and "heavy rain and flood warn- 
ing", or generally undesirable keywords are memorized 
as risk information keywords, and if such risk informa- 
tion keywords are found in a piece of information auto- 
matically collected during autonomous behavior, the 40 
piece of information can be designed to be displayed as 
absolute behavior. As described above, by autono- 
mously detecting risk information and setting the infor- 
mation as absolute behavior, the risk information can be 
given priority over other information and be displayed in 45 
addition to an action plan, when the user's action is trav- 
elling. 

Behavior Planning Unit 

50 

[0081] The behavior pfenning urrif ttrecerves behavior 
information from each of the emotion expression deci- 
sion unit D, the autonomous behavior generation unit E, 
and the commanded behavior generation unit F; sets 
degrees of priority at each item of behavior information ss 
based on the pseudo-personality information, the 
pseudo-emotion information, and the user information; 
generates new priority behavior information; combines 



each item of behavior information based on the new pri- 
ority behavior information having priority; and decides 
final output behavior. 

[0082] Figures 10 and 1 1 are schematic flow charts 
showing processes at the behavior planning unit H. 
[0083] The behavior planning unit H first judges 
whether or not there is required absolute behavior. At 
the time, if there is required absolute behavior, the 
absolute behavior information is set at priority 1. The 
absolute behavior information includes information on 
absolute behavior generated at the commanded behav- 
ior generation unit as described above, and information 
on behavior requiring urgent action against a situation 
such as input device failura 
[0084] As the result of judgement as to whether there 
is required absolute behavior, if there is no required 
absolute behavior, a degree of priority (1 to 3: in this 
case, priority is 1, La, high priority) is determined for 
each item of behavior information based on a table of 
degrees of priority, and is set as new priority behavior 
information. 

[0085] The table of degrees of priority is a table defin- 
ing degrees of priority as described in Figure 10, for 
example: If the pseudo-emotion is "joy", the pseudo- 
personality is "obedience", and the state of the user is 
"anger", the commanded behavior is set at priority 1 , the 
emotion expression behavior is set at priority 2, and the 
autonomous behavior is set at priority 3. As above, the 
table is a group of data wherein a degree of priority of 
each item of behavior information is set for a combina- 
tion of ail pseudo-emotions, all pseudo-personalities, 
and all user's states. However, the method of determin- 
ing the degree of priority is not limited to the above, and 
includes, for example, fuzzy logic technique using fuzzy 
rules which define in advance the relationship between 
the pseudo-emotions, the pseudo-personalities, the 
user's states, and the degrees of priority of behavior 
information, wherein membership equations of behavior 
information are formulated with respect to the deg-ees 
of priority. 

[0086] After setting the new priority behavior informa- 
tion, the behavior which has already been performed 
and the behavior which need not be performed are 
deleted, and the remaining behavior patterns are set as 
incomplete behavior information. This incomplete 
behavior information is set to be more important than 
the aforesaid new priority behavior information. Based 
on the incomplete behavior information and the new pri- 
ority behavior information, the current priority behavior 
information is updated. Each item of behavior informa- 
tion is combined based on the current priority behavior 
information. 

[0087] Figure 1 1 is a schematic flow chart showing 
processes from generation of each item of behavior 
information based on the current behavior information 
to setting of final output behavior. 
[0088] Composite behavior is created by selling a 
behavior having a higher degree of priority first. That is, 
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each item of behavior information is composed of plural 
behaviors, and in practice, the behavior information of 
this embodiment includes any of the following: eyes, 
mouth, conversation, display, search, and file. In order 
of priority, behaviors in the behavior information are set, 5 
and a behavior in the behavior information having lower 
priority is set only when there is no conflict with the 
behaviors already defined by the higher priority behav- 
ior information, i.e., the behaviors which the high priority 
behavior information does not define can be filed with w 
those in the lower priority behavior information. 
[0089] In the embodiment indicated in Figure 1 1, the 
behavior (display: newly released information) defined 
by the commanded behavior which has the highest pri- 
ority is set first Second, the behavior defined by the is 
emotion expression behavior having priority 2 (eyes: 
smiling eyes; mouth: smiling mouth; conversation: whis- 
tling) is set. Third, the behavior defined by the com- 
manded behavior having priority 3 (search: hot spring) 
is set Fourth (last), the behavior defined by the autono- 20 
mous behavior having priority 4 is set As can be seen 
in the above example, although the autonomous behav- 
ior having priority 4 defines "conversation: chatting (type 
3)", the behavior has been defined by the emotion 
expression behavior having priority 2 (conversation: 25 
whistling), i.a, "conversation" has already been defined. 
Thus, the behavior "conversation" defined by the auton- 
omous behavior is not adopted. In the above, the rea- 
son that there are two commanded behaviors with 
regard to "conversation" in the above example is that 30 
"commanded behavior information" in the previous 
incomplete behavior was maintained, i.e., the previous 
incomplete behavior information is assigned priority. 

Display Screen as 

[0090] Figures 22 and 23 are schematic view of a dis- 
play screen of the touring assist system according to the 
present invention. Figure 22 shows a main screen, and 
Figure 23 shows a second screen. 40 
[0091] This system can be installed in a car or can be 
in the form of a lap-top computer. This system can be 
operated by punching a keyboard, touching a display, or 
by verbal instructions. A face displayed at the upper left 
comer shows the current emotion of the system. A 45 
panel displayed on the left shows the current states of 
the user, the system, and the environment, and the 
user's character (personality) and the system's charac- 
ter (personality). Since this system is not used for a sig- 
nificant time period by this user, the system has not yet so 
developed full pseirfo-pereonaffty: Fbr c u f fec t fiTg infor- 
mation related to the environment, this system is 
equipped with a light sensor, a temperature sensor, a 
pressure sensor, and auditory sensor, whose readings 
are indicated on the left panel. Fbr collecting information 55 
related to the user, the system is equipped with a posi- 
tion sensor, auditory sensor, and touch detection sen- 
sor, whose readings are indicated on the left panel. The 



system is also equipped with a communication tool to 
access external databases or external information 
sources. Output behavior of this system includes facial 
expression (upper left comer), written message (top on 
the right panel), indication of information (the right 
panel), and voice messaga In Figure 22, the user wants 
to make a travel plan for a vacation and sits in front of 
the system. The system asks the user how the user is 
verbally as well as on the display. The user's response 
is quick and the user is fine, resulting in creation of 
happy feeling in the system as indicated in the face at 
the corner. The system asks the user what the request 
is. The user requests searching for a place to visit, and 
accordingly, the system shows options on the display as 
shown in Figure 23. The user does not answer the sys- 
tem (he cannot decide), and as a result the system 
shows a neutral face changed from the happy face, ff 
the user does not do anything particular, the system will 
obtain a patient character (will show "obey" on the dis- 
play) and expression thresholds of emotions will be high 
(passive), ff the user finds a place he wants to go and 
shows his, feeling (happy), but cannot schedule it for 
some reason, the system will show sad feeling and sug- 
gest an alternative schedule the user is likely to find 
agreeable. The system will eventually obtain characters 
similar to those of the user. 

Effects 

[0092] As described above, according to this assist 
system, the assist system itself is equipped with a 
pseudo-personality, which grows by interacting with the 
state of the user, and pseudo-emotions, which change 
in accordance with the state of the user and the external 
information, wherein the pseudo-emotions are control- 
led by the pseudo-personality, and the pseudo-person- 
ality is different depending on the user, La, the pseudo- 
personality unique to each user is being established in 
accordance with the use of the system. Since the 
pseudo-personality is expressed as the pseudo-emo- 
tions, the user can enjoy fostering a unique assist sys- 
tem during a growth period, similar to raising a child. 
Further, approximately after completion of the growth 
period, the user can enjoy establishing "friendship" with 
the assist system. 

[0093] Further, the system is constituted so as to gen- 
erate autonomous behavior based on the pseudo-per- 
sonality and the pseudo-emotions controlled by the 
pseudo-personality. Thus, the more the system is used, 
the more the understanding of the user's feelings or 
preferences can be achieved. The system can assist 
the user in matters which the user does not directly 
request. 

[0094] Further, since the assist system can perform 
both autonomous behavior and commanded behavior, 
the user need not give detailed commands from the 
beginning. The user can set conditions through dia- 
logue between the system and the user. In addition, dur- 
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ing the dialogue, the assist system can give its own 
suggestions (such as newly released information). 
Thus, an appropriate action plan can be established 
very easily with the assist system. Particularly, if the 
new behavior pattern generation unit is activated to gen- 5 
erate new behavior patterns (including new emotion 
expression patterns), it is not necessary to prepare 
numerous behavior patterns in advance in combination 
with the pseudo-personality, the pseudo-emotions, and 
the other information (user/external information). The w 
system can respond to any type of user more quicWy, 
naturally and adaptively. 

Pet Robot 

15 

[0095] With reference to Figure 12-21, another 
embodiment to which the control system of the present 
invention is adopted will be explained. This embodiment 
is a pet robot which a user can keep as a pet. 
[0096] Incidentally, as can be seen in Figures 12-21 , 20 
the control system of the pet robot is almost the same 
as that in the assist system, except that the details of 
information at each processing unit, output devices, and 
the details of output information are different Thus, the 
principle of the control system will not be explained 25 
again to avoid repetition. The different aspects from the 
asset system-will be explained below. 
[0097] The pet robot is designed to give a user para- 
experience of keeping a pet. The pet robot itself has a 
pseudo-personality, which grows by interacting with the 30 
user and through its own experience, and pseudo-emo- 
tions, which change in accordance with interaction with 
the user and the surrounding environments. Depending 
on the state of the pseudo-personality and the pseudo- 
emotions, the pet robot expresses emotions and 35 
responds to the user with autonomous response and/or 
commanded response As a result, the pet robot is 
capable of conducting various actions. 
[0098] In practice, in order to recognize the state of 
the user or the environments, this pet robot is equipped 40 
with a visual detection sensor, a touch detection sensor, 
a hearing-detection sensor, and a means for accessing 
external databases, and further equipped with an output 
device, such as a visual output device and an auditory 
output device, for indicating its pseudo-emotions or 45 
responses to the user, and further with traveling means 
functioning as its hands and feet, an operation device, 
and a tail. 

Basic Structures of Pet Robot so 

[0099] Figure 1 2 is a schematic block diagram show- 
ing an example of the entire system of a pet robot 
according to the present invention. As shown in this fig- 
ure, the pet robot comprises: a user/environment infor- 55 
mation recognition unit A for recognizing a state of the 
user or environment information; a pseudo-personality 
information-setting unit B for setting a pseudo-personal- 



ity by referring to the outcome of recognition by the rec- 
ognition unit A; a pseudo-emotion generation unit C for 
generating pseudo-emotions based on the outcome of 
recognition by the recognition unit A and the information 
on the pseudo-personality set by the pseudo-personal- 
ity information-setting unit B; an emotion expression 
decision unit D for deciding manners of emotion expres- 
sion, such as expression methods to express laughing 
or angry on a display, based on the information on the 
pseudo-emotion; an autonomous behavior generation 
unit E for generating autonomous behavior based on 
the outcome of recognition, the information on the 
pseudo-personality, and the information on the pseudo- 
emotions; a commanded behavior generation unit F for 
generating behavior in accordance with the user's com- 
mand based on the outcome of recognition and at least 
the preceding behavior previously outputted by this sys- 
tem itself; a new behavior pattern generation unit G for 
generating new behavior patterns which are usable in 
the emotion expression decision unit D and the autono- 
mous behavior generation unit E based on the pseudo- 
personality information, the pseudo-emotion informa- 
tion, and a database storing various behavior elements 
and emotion expression elements; and a behavior plan- 
ning unit H for deciding final behavior to be outputted 
using output from each of the (emotion expression deci- 
sion unit D, the autonomous behavior generation unit E, 
and the commanded behavior generated unit F, based 
on the outcome of recognition, the information on the 
pseudo-personality, and the information on the pseudo- 
emotions. As shewn in Figure 13, once the system is 
activated, this assist system repeats each processing 
step at units A through H until the system is turned off. 
The structure and function of each unit will be explained 
in detail below. 

User/Environment Information Recognition Unit 

[0100] As shown in Figure 12, the user/environment 
information recognition unit A comprises various input 
devices (e.g., an auditory input device, a visual input 
device, a touch detection device, and/or an environment 
information detection device), a user's evaluation recog- 
nition unit Al, a user's emotion recognition unit A2, a 
user recognition unit A3, a user's command recognition 
unit A4, an environment information recognition unit A5, 
and a current user information/characteristics deduction 
unit A6. The user's evaluation recognition unit Al per- 
forms recognition-processing based on the information 
obtained via the input devices. The current user infor- 
matfon/tftaracteristics deduction unit A6 determines or 
deduces a current state of the user and the user's char- 
acteristics such as the user's preferences based on the 
recognition outcome from the user-related recognition 
units Al through A3. 

[0101] In the above, the input device includes a micro- 
phone as an auditory input device, a camera as a visual 
input device, a tactile sensor as a touch detection 
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device, and an illumination sensor, a voltage sensor, an 
obstacle-sensing sensor, and a drive-sensing sensor 
(sensing a distance driven by a drive device as an envi- 
ronment information detection device. However, the 
input devices are not limited to the above. Any input 5 
device can be used which is available on the market, 
and which is capable of detecting the user's state and 
environment information visually, auditorily, and/or tac- 
tile!* 

[0102] Figure 14 is a schematic block diagram show- 10 
ing the user/environment information recognition unit A 
in detail to explain processes at each of the processing 
units Al through A6 in the user/environment information 
recognition unit A. 

[0103] As shown in this figure, the user's evaluation is 
recognition unit Al establishes the user's evaluation as 
to behavior outputted by the pet robot and sets the 
result, for example, as information on the user's evalua- 
tion, based on the manner in which the user responds to 
the pet robot. The manner may be sensed by a tactile 20 
sensor, ag., how the user touches or strokes the pet 
robot. If the manners of touching or stroking are corre- 
lated with evaluation values in advance, the evaluation 
values can be determined. 

[0104] The user's emotion recognition unit A2 pre- 25 
sumes the user's current emotions based on the char- 
acteristics found in the auditory information and the 
user's facial expression, wherein an auditory analysis 
(analyse of frequency distributions or pitches of the 
sound/voice) and a facial expression analysis (analysis 30 
based on the shapes of the eyes and the mouth) are 
conducted based on the auditory information and the 
user's facial expression inputted from the auditory input 
device and the visual input device. Further, based on 
the presumed emotions, the user's emotion recognition 35 
unit A2 determines the user's current emotion. 
[0105] The user recognition unit A3 recognizes 
whether the person detected by the pet robot is the 
user, based on the outcome of recognition of the posi- 
tion of the person and the body-type of the person by. 40 
using a visual input device, and based on the outcome 
of analysis of the sound/voice obtained at the user emo- 
tion recognition unit A2. 

[0106] The user's command recognition unit A4 rec- 
ognizes and establishes the user's command irrforma- 45 
tion based on command information obtained by 
analyzing the meaning of the sound/voice recognized 
by the auditory input device and by analyzing informa- 
tion on the user's gestures recognized by the visual 
input device. For example, if the user says "fetch" and so 
"points out a ball wth Msrtter gesture, Based on a com- 
bination, the user's command information is set as fol- 
lows: The command is "fetching" and the object is "the 
ball". 

[0107] The environment information recognition unit ss 
AS recognizes the location of an object based on infor- 
mation obtained from the visual input device; recog- 
nizes the travelling route of the pet robot when travelling 



autonomously, based on information obtained from the 
drive-sensing sensor of the drive device; converts infor- 
mation, such as brightness, the presence or absence of 
an obstacle, and the electrical power level of the pet 
robot, obtained from various detection sensors, to 
appropriate forms; sets the converted information; and, 
based thereon, sets information on the environments 
surrounding the pet robot and the internal state of the 
pet robot. 

[0108] The current user information/charactenstics 
deduction unit A6 determines or deduces the current 
state and the characteristics of the user based on the 
recognition results at the user-related information rec- 
ognition units A1 through A4, the information on the cur- 
rent priority behavior described later, the information on 
the emotion expression patterns, and the information on 
the autonomous behavior patterns. The current user 
informationAc^iaracteristics deduction unit A6 then 
updates data of the recognition information related to 
the user stored in a user information memory 1. 
[0109] In the above, the current state of the user 
includes the level of the user's emotions, the user's eval- 
uation as to the behavior outputted by the system, and 
the location of the user. The characteristics of the user 
includes frequency of the occurrence of each emotion, 
the change rate or range of emotions, and the average 
value of user's evaluation as to the behavior outputted 
by the system. 

[01 1 0] The output of the user's command recognition 
unit A4 is transmitted to a user's command information 
memory 2. thereby updating the information related to 
the user's command stored in the memory 2. The infor- 
mation related to the user's command includes, for 
example, the contents of the command, the object of the 
command, and the conditions of the command. 
[0111] The output of the external information recogni- 
tion unit AS is transmitted to an external information 
memory 3, thereby updating the information stored in 
the memory 3. 

Pseudo-Personality Information-Setting Unit 

[01 1 2] Figure 15 is a schematic flow chart showing 
processes at the pseudo-personality information-setting 
unit B. 

[0113] The pseudo-personality information-setting 
unit B is constructed so as to set as a pseudo-person- 
ality, a group of several preliminary personalities, each 
subdividable into several levels. Based on the informa- 
tion stored in the user information memory 2, the level of 
each preffrnfhary personality is set, and the higher the 
level, the more the preliminary personality contributes to 
the formation of the pseudo-emotions and the action 
plan. 

[0114] As can be seen, processes at the pseudo-per- 
sonality information-setting unit B are substantially the 
same as those in the assist system. The details of the 
processes are therefore omitted. 
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Pseudo-Emotion Generation Unit 

[0115] Figure 16 is a schematic flow chart showing 
processes at the pseudo-emotion generation unit C. 
[0116] The pseudo-emotion generation unit C s 
receives output from the user information memory 1 , the 
external information memory 3, and the pseudo-person- 
ality information memory 4, and, based on the above 
information, the pseudo-emotion generation unit C gen- 
erates pseudo-emotion information of the system. 10 
[0117] As can be seen, processes at the pseudo-emo- 
tion generation unit C are substantially the same as 
those in the assist system. The details of the processes 
are therefore omitted. 

15 

Emotion Expression Decision Unit 

[0118] Figure 17 is a schematic flow chart showing 
processes at the emotion expression decision unit D. 
[01 1 9] The emotion expression decision unit D judges 20 
the level of each preliminary emotion from the pseudo- 
emotion information obtained from the pseudo-emotion 
memory 5, and determines emotion expression to be 
outputted based on preliminary emotions whose levels 
are high or more than zero. 25 
[01 20] The emotion expression is selected from plural 
patterns of emotion expression in relation to pseudo- 
emotions stored in advance in an emotion expression 
pattern memory 6. The emotion expression includes, in 
practice, changes in shape of parts such as eyes and a 30 
mouth if a lace" is created by a visual output device. 
Further, emotion expression can be indicated by move- 
ment of a tail or the pet robot rtsetf. In the above, the 
emotion expression pattern memory 6 memorizes in 
advance patterns of each of parts of "face", and pat- 35 
terns of movement of the tail or the pet robot itself in 
relation to each level of each preliminary emotion. 
[0121] The method of selecting the patterns of the 
emotion expressions at the emotion expression unit D 
can be simply a method wherein the pattern of the emo- 40 
Hon expression, which corresponds to the preliminary 
emotion having the highest level, is selected. However, 
preferably, a pattern of the absence of expression 
(expressionless) is created among patterns of emotion 
expressions, and if the levels of all of the preliminary 45 
emotions are zero, the expressionless pattern is 
selected, and if the levels of preliminary emotions, 
except for one preliminary emotion, are zero, a pattern 
corresponding to the one preliminary emotion is 
selected, if the levels of more than one but not all prelim- so 
inary emotions are more than zero, a pattern of a pre- 
liminary emotion having a higher level than those of the 
other preliminary emotions has priority over the other 
patterns. Further, among the remaining patterns, a pat- 
tern of a preliminary emotion having a lower level than 55 
the first one is selected where the first pattern does not 
apply. In the above, the first pattern and the second pat- 
tern are combined to make one combined expression. If 



the levels of an of the preliminary emotions are more 
than zero, the expressionless pattern may be selected. 
Accordingly, the pseudo-emotions can be indicated in a 
complex manner. 

[01 22] The emotion expression decision unit D deter- 
mines emotion expression patterns from the pseudo- 
emotions at the moment in the above-described man- 
ner, sets the patterns as emotion expression behavior, 
and outputs them to the behavior planning unit H. 

Autonomous Behavior Generation Unit 

[0123] Figure 18 is a schematic flow chart showing 
processes at the autonomous behavior generation unit 
E. 

[0124] The autonomous behavior generation unit E 
determines each action level of plural itemized autono- 
mous behaviors set in an autonomous behavior pattern 
memory 7 based on each level of the environment infor- 
mation, the user information, the pseudo-emotion infor- 
mation, and the pseudo-personality information, in 
order to determine a degree of priority of each itemized 
autonomous behavior. In the above, each of the envi- 
ronment information, the user information, the pseudo- 
emotion information, and the pseudo-personality infor- 
mation is subdvided into plural levels. 
[01 25] In this embodiment, the itemized autonomous 
behavior patterns are prepared for all of changeable or 
movable parts (such as the eyes, the mouth, voices, the 
height of the end of the tail, the movement of the tail, 
and the travelling of the pet robot itself). Each itemized 
autonomous behavior is correlated with the levels of 
each item of information By adding or subtracting 
action levels of each itemized autonomous behavior 
based on the level of each item of information, the 
action level of each itemized autonomous behavior is 
determined. 

New Behavior Pattern Generation Unit 

[01 26] Both of the emotion expression pattern infor- 
mation memory 6 and the autonomous behavior pattern 
memory 7, which are used in the emotion expression 
decision unit D and the autonomous behavior decision 
unit E, are updated at the new behavior pattern genera- 
tion unit Q at appropriate intervals. 
[0127] Figure 19 is a schematic flow chart showing 
processes at the new behavior pattern generation unit 
G indicated in Figure 12. 

[0128] The new behavior pattern generation unit G 
judges ftitervate Between generation of new behavior 
patterns and subsequent generation of new behavior 
patterns based on the pseudo-personality information. 
In practice, the levels of "emotion fluctuation" or 
"aggressiveness", for example, in the pseudo-personal- 
ity information are correlated with intervals of genera- 
tion. For example, correlation is formulated in such a 
way that the higher the level of "aggressiveness", the 



16 



31 



EP0 978770A2 



32 



shorter the interval becomes. 
[0129] The new behavior pattern generation unit Q 
calculates intervals of generating new behavior pat- 
terns, and then judges whether generation off new 
behavior patterns is activated based on the intervals s 
and random elements. If generation is judged to be acti- 
vated, the new behavior pattern generation unit G com- 
bines behavior elements stored in an emotion 
expression element database 8 and an autonomous 
behavior element database 9, which memorize in w 
advance a number of emotion expression patterns and 
autonomous behavior patterns, respectively, thereby 
setting new emotion expression patterns and new 
autonomous behavior patterns. The above combining 
processes can be conducted randomly or by using is 
genetic algorithms. In practice, for example, various 
shapes of portions of a "face" such as eyes, various 
movements of a tail, various movements of the pet robot 
itself, and various types of utterance are inputted as 
emotion expression elements in the database in 20 
advance, and new expression patterns of "anger can 
be generated randomly using emotion expression ele- 
ments stored in the database. In the above, the behav- 
ior patterns can include emotion expression patterns, 
i.e., simply one integrated memory can store these 25 
pieces of information, and processes for new emotion 
expression patterns and processes for new autono- 
mous behavior patterns can be integrated. 
[01 301 After generating new emotion expression pat- 
terns and new autonomous behavior patterns according 30 
to the above process, these new patterns are checked 
and revised with a common sense database and the 
user information to eliminate extreme or obviously dis- 
taste patterns. In practice, based on the common sense 
database including information such as "when express- ss 
ing 'anger*, the corner of the eye is not down", the new 
patterns are checked and revised with respect to com- 
mon sense aspects. Further, if when expression of 
"anger is associated with "growling", the user 
expresses uncomfortable reaction, the sound associ- ao 
ated with "anger is changed from loud growling" to 
"quiet growling for example. Checking based on the 
user's response or evaluation is conducted using the 
user information. 

[0131] After the new behavior pattern generation unit as 
G checks and revises the newly generated patterns as 
described above, the unit G temporarily memorizes the 
currently used behavior patterns corresponding to the 
newly generated patterns, and then temporarily updates 
the current behavior patterns to the new behavior pat- so 
terns. These temporary new Btgftdviur pa t terns are eval- 
uated. 

[01 32] Evaluation of the new behavior patterns is con- 
ducted mainly based on the user's evaluation in the user 
information. If evaluations of the new behavior patterns ss 
are not good after using the new behavior patterns for a 
given period of time, the memorized previous behavior 
patterns are resumed. If the result of evaluation is posi- 



tive, the new behavior patterns are memorized in the 
respective pattern information memoirs 6 and 7. 

Commanded Behavior Generation Unit 

[0133] The commanded behavior generation unit F 
generates commanded behavior by using information 
from the environment information memory 3 in accord- 
ance with the user's command included in the user's 
command information. In practice, in this embodiment if 
the user's commanded information is (command: fetch- 
ing; object ball), the unit F generates behavior informa- 
tion including recognition of the location of the ball 
based on information from the environment information 
memory, moving to the location, fetching the ball, and 
returning to the user. 

Behavior Planning Unit 

[9134] The behavior planning unit H receives behavior 
information from each of the emotion expression deci- 
sion unit D, the autonomous behavior generation unit E, 
and the commanded behavior generation unit F; sets 
degrees of priority at each item of behavior information 
based on the pseudo-personality information, the 
pseudo-emotion information, and the user information; 
generates new priority behavior information; combines 
each item of behavior information based on the new pri- 
ority behavior information having priority; and decides 
final output behavior. 

[0135] Figures 20 and 21 are schematic flow charts 
showing processes at the behavior planning unit H. 
[0136] The behavior planning unit H first judges 
whether or not there is required absolute behavior. At 
this time, if there is required absolute behavior, the 
absolute behavior information is set at priority 1. The 
absolute behavior information includes information on 
behavior requiring urgent action against a situation such 
as battery power shortage and arrest by collision. The 
conditions corresponding to the absolute behavior are 
memorized in the behavior generation unit H. 
[01 37] As the result of judgement as to whether there 
is required absolute behavior, if there is no required 
absolute behavior, a degree of priority (1 to 3: in this 
case, priority is 1, i.e.. high priority) is determined for 
each item of behavior information based on a table of 
degrees of priority, and is set as new priority behavior 
information. 

[01 38] The table of degrees of priority is a table defin- 
ing degrees of priority as described in Figure 20, for 
example: ff the pseudo-emotion is Iqy"; the pseudo- 
personality is "obedience", and the state of the user is 
"anger", the commanded behavior is set at priority 1 , the 
emotion expression behavior is set at priority 2, and the 
autonomous behavior is set at priority 3. As above, the 
table is a group of data wherein a degree of priority of 
each item of behavior information is set for a combina- 
tion of all pseudo-emotions, all pseudo-personalities, 
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and all user's states. However, the method of determin- 
ing the degree of priority is not limited to the above, and 
includes, for example, fuzzy logc technique using fuzzy 
rules which define in advance the relationship between 
the pseudo-emotions, the pseudo-personalities, the 5 
user's states, and the degrees of priority of behavior 
information, wherein membership equations of behavior 
information are formulated with respect to the degrees 
of priority. 

[01 39] After setting the new priority behavior informa- 10 
tion, the behavior which has already been performed 
and the behavior which need not be performed are 
deleted, and the remaining behavior patterns are set as 
incomplete behavior information. This incomplete 
behavior information is set to be more important than is 
the aforesaid new priority behavior information. Based 
on the incomplete behavior information and the new pri- 
ority behavior information, the current priority behavior 
information is updated. Each item of behavior informa- 
tion is combined based on the current priority behavior 20 
information. 

[0140] Figure 21 is a schematic flow chart showing 
processes from generation of each item of behavior 
information based on the current behavior information 
to setting of final output behavior. 2s 
[0141] Composite behavior is created by setting a 
behavior having a higher degree of priority first That is, 
each item of behavior information is composed of plural 
behaviors, and in this embodiment indicated in Figure 
21, the behavior information of this embodiment 30 
includes any of the following: eyes, mouth, conversa- 
tion, travelling, and information. In the order of priority, 
behaviors in the behavior information are set and a 
behavior in the behavior information having lower prior- 
ity is set only when there is no conf Oct with the behav- 35 
iors already defined by the higher priority behavior 
information, i.e., the behaviors which the high priority 
behavior information does not define can be f flled with 
those in the lower priority behavior information. 
[0142] In the embodiment indicated in Figure 21, the 40 
behavior (travelling: approaching the user) defined by 
the commanded behavior which has the highest priority 
is set first Second, the behavior defined by the emotion 
expression behavior having priority 2 (eyes: smiling 
eyes; mouth: smiling mouth; conversation: whistling; 45 
travelling: swaying the body) is set. Third, the behavior 
defined by the commanded behavior having priority 3 
(information: indicating the remaining battery power 
level) is set Fourth (last), the behavior defined by the 
autonomous behavior having priority 4 (conversation: so 
whistling; travelling: movfngto a brighter ptace) te set 
As can be seen in the above example, although the 
autonomous behaviors having priorities 1 , 2, and 4 have 
information on "travelling", only the information on "trav- 
elling" in the behavior having priority 1 is employed, and ss 
the information on Travelling" in the behavior having 
lower priorities is not adopted. In the above, the reason 
that there are two commanded behaviors with regard to 



travelling" in the above example is that "commanded 
behavior information" in the previous incomplete behav- 
ior was maintained, i.e., the previous incomplete behav- 
ior information is assigned priority. 

Effects 

[0143] As described above, according to this system, 
the pet robot itself is equipped with a pseudo-personal- 
ity, which grows by interacting with the state of the user, 
and pseudo-emotions, which change in accordance 
with the state of the user and the environment informa- 
tion, wherein the pseudo-emotions are controlled by the 
pseudo-personality, and the pseudo-personality is dif- 
ferent depending on the user La, the pseudo-personal- 
ity unique to each user is being established in 
accordance with the use of the pet robot Since the 
pseudo-personality is expressed as pseudo-emotions, 
the user can enjoy fostering a inique pet robot during a 
growth period, similar to raising a child. Further, approx- 
imately after completion of the growth period, the user 
can enjoy establishing "friendship" with the pet robot 
[0144] Further, the pet robot is constituted so as to 
generate autonomous behavior based on the pseudo- 
personality and the pseudo-emotions controlled by the 
pseudo-personality. Thus, the more the pet robot is 
used, the more the personality of the pet robot is estab- 
lished, thereby allowing the pet robot to behave in vari- 
ous manners. 

[0145] Further, since the pet robot can perform both 
autonomous behavior and commanded behavior, the 
pet robot will never be completely delinquent nor com- 
pletely obedient Various actions of the pet robot keep 
the user interest in the pet robot Particularly, if the new 
behavior pattern generation unit is activated to generate 
new behavior patterns (including new emotion expres- 
sion patterns), it is not necessary to prepare numerous 
behavior patterns in advance in combination with the 
pseudo-personality, the pseudo-emotions, and the other 
information (user/external information). The system can 
respond to any type of user more quickly, naturally, and 
adaptivery. 

Technical Options 

[0146] A robotics platform suitable for mobile naviga- 
tion tasks can be designed to provide on-board 
audio/video capabilities and a rado link to an off-board 
computer. Existing techniques for color tracking to track 
the user's hands and head location can be adapted. 
Existing word spotting speech recognition software for 
recognizing simple one- or two-word spoken commands 
can also be adapted. Any methods for simple non-ver- 
bal audio analysis, such as verbal intonation analysis to 
recover the affective or emotion content of a command 
can be adapted. Existing schema-based reactive navi- 
gation methods can be adapted to provide the system 
with the capability of navigating around its environment 
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without colliding with obstacles. In the above, explora- 
tion behaviors as well as goaJ seeking behaviors for 
moving to specific destinations can be encoded. Fur- 
ther, "reasoning" capabilities and a repertoire of behav- 
iors can be formed with an appropriate decision-making 5 
algorithm that takes into account the system's goals and 
goal priorities, the current environment or situation, and 
user's goals and emotion state, for exampla Finally, any 
algorithms can be adapted for adapting the system's 
behavior in response to unexpected situations, for w 
learning new behaviors, for learning the environment 
and for learning about the user's preferences. 
[0147] The functioning of the system can be organized 
by, for example, expanding a behavior-based subsys- 
tem (responsible for low-level" interaction of the system is 
with the environment) with high-level processing mod- 
ules (responsible for "high-level" interaction of the sys- 
tem with the user) that work in parallel. 
[0148] Further, correlations between various informa- 
tion and various variables can be determined in 20 
advance using existing techniques such as neural net- 
works, fuzzy neural networks, and genetic algorithms if 
the correlations are highly complex, or using existing 
techniques such as maps and functional equations if the 
correlations are rather simple. In this regard, Da Ruan 2s 
(editor) "Intelligent Hybrid Systems - Fuzzy Logic, Neu- 
ral Networks, and Genetic Algorithms -" Wuwer Aca- 
demic Publishers (1997), J.-S. R. Jang, C.-T. Sun, E. 
Mizutani, "Neuro-Fuzzy and Soft Computing" Prentice 
Hall Upper Saddle River, NJ 07458 (1997), C.-T. Lin and 30 
C. S. George Lee, "Neural Fuzzy Systems" Prentice 
Hall Upper Saddle River, NJ 07458 (1998), and N. K. 
Kasabov, "Foundations of Neural Networks, Fuzzy Sys- 
tems, and Knowledge Engineering" the MIT Press 
(1996) are hereby incorporated by reference. The above as 
techniques can be combined, and learning control can 
be adapted for any techniques. 
[0149] Generation of new behavior patterns is impor- 
tant to achievement of high and quick adaptivity, in par- 
ticular based on information of user or environment 40 
around the robot The generation is preferably con- 
ducted using genetic algorithms. Genetic algorithms 
can be applied to quickly evolve any predetermined 
relationships or combinations or priorities, such as 
those stored in the pseudo-personality forming unit, the 45 
pseudo-emotion forming unit, the autonomous behav- 
ior-establishing unit, and the behavior control unit in 
order to "foster" an object. 

[01 50] The data on the pseudo-personality, for exam- 
ple mental information or intelligence information can be so 
saved in a separable memory medftnn sucfrasa ram or * 
compact disc, so that the user can implant the pseudo- 
personality in another object. In addition, the data can 
be transmitted to another object via communication 
means. By using the above technology, plural pseudo- ss 
personalities of plural objects can be hybridized or 
crossbred to create a new pseudo-personality. Further, 
the intelligent portions of the system such as the 



pseudo-personality forming unit, the pseudo-emotion 
forming unit the autonomous behavior-establishing 
unit the commanded behavior-establishing unit the 
behavior control unit, the other processing units, and 
memories, can be installed in a main computer sepa- 
rately from the object, wherein a network is established 
between the main computer and the object via, for 
example, the Internet so that the object can be made 
compact and which may need to have simply sensing 
devices, an output device, and a communication device 
for contacting the main computer. 
[01 51] One possfoility to realize this is a behavior con- 
trol system which has a behavior unit programmed to 
behave based on behavior parameters. Moreover, this 
system includes an input unit programmed to output sig- 
nals when receiving signals resulting from the behavior. 
Furthermore, there is included a memory medium 
attachable between the behavior unit and the input unit 
and detachable therefrom. That memory medum stor- 
ing a relationship between the signals inputted from the 
input unit and behavior parameters outputted to the 
behavior unrt, said relationship programmed to be 
updated based on the signals from the input unit 
[0152] Said memory medium may be included in a 
personality forming unit which is programmed to deter- 
mine a relationship between signals inputted from the 
input unit and behavior parameters outputted to the 
behavior unit and which is programmed to update the 
relationship based on the signals for the input unit 
stored in the memory mecfium. 
[0153] The above-mentioned behavior control system 
further may comprise an emotion forming unit provided 
between the input unit and the behavior unit in parallel 
to the personality forming unit Said emotion forming 
unit is programmed to determine a relationship between 
signals inputted from the input unit and second behavior 
parameters added to the first behavior parameters and 
outputted to the behavior unit, based on the signals 
from the input unit. 

[0154] Moreover, this behavior control system may 
further comprise a new behavior generation unit pro- 
vided between the input unit and the behavior unit in 
parallel to the memory medium. Said new behavior gen- 
eration unit is programmed to generate new patterns of 
behavior randomly or under predetermined rules and to 
output the new patterns to the behavior unit 
[0155] The above-mentioned network may be a net- 
work of a plurality of autonomic devices, each behaving 
adaptrvely to a user. This network comprises a sensing 
unit which senses preselected signals from the user 
who uses the autonomic device. Moreover, there is 
included a behavior unit programmed to behave based 
on behavior parameters. Further, there is provided an 
input unit programmed to output signals when receiving 
signals resulting from the behavior through the sensing 
unit and signals provided through the network. In addi- 
tion, there is provided a behavior parameter forming unit 
provided between the behavior unit and the input unit. 
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Said behavior parameter forming unit comprising a 
memory storing a relationship between signals inputted 
from the input unit and behavior parameters outputted 
from the behavior unit. Moreover, said behavior param- 
eter forming unit is programmed to update said relation- 5 
ship based on the signals from the input unit wherein 
each device is connected with other devices through the 
network. 

[0156] In this network, each device further comprises 
an emotion forming unit provided between the input unit w 
and the behavior unit in parallel to the behavior param- 
eter forming unit Said emotion forming unit is pro- 
grammed to determine a relationship between signals 
inputted from the input unit and second behavior param- 
eters added to the first behavior parameters and output- is 
ted to the behavior unit, based on the signals from the 
input unit 

[0157] Moreover, each device further comprises a 
new behavior generation unit provided between the 
input unit and the behavior unit in parallel to the behav- 20 
ior parameter forming unit Said new behavior genera- 
tion unit is programmed to generate new patterns of 
behavior randomly or under predetermined rules and to 
output the new patterns to the behavior unit. 
[0158] A further possibility of a network is a network of 2s 
plurality of autonomic devices, each behaving adap- 
tively to the user. This network comprises a sensing unit 
which senses preselected signals from a user who uses 
the autonomic device. Further there is provided a 
behavior unit programmed to behave based on behavior 30 
parameters and an input unit programmed to output sig- 
nals when receiving signals resulting from the behavior 
through the sensing unit and signals provided through 
the network. Said network further comprising a main 
computer connecting each device through the network. 35 
Said computer comprising a behavior parameter form- 
ing unit provided between the behavior unit and the 
input unit of each device through the network. Moreover, 
said behavior parameter forming unit comprising a 
memory storing a relationship between signals inputted 40 
from the input unit and behavior parameters outputted 
to the behavior unit of each device. Said behavior 
parameter forming unit is propammed to update said 
relationship of each device based on the signals from 
the input unit, whereas said computer receiving the sig- 45 
nals from the sensing unit or the input unit 
[0159] Said computer may further comprise an emo- 
tion forming unit provided between the input unit and the 
behavior unit in parallel to the behavior parameter form- 
ing unit of each device through the network. Saidemo- so 
tion forming unit is programmed to determrrre a 
relationship between signals inputted from the input unit 
and second behavior parameters added to the first 
behavior parameters and outputted to the behavior unit 
of each device, based on the signals from the input unit 55 
of each device. 

[0160] In addtion, said computer further may com- 
prise a new behavior generation unit provided between 



the input unit and the behavior unit in parallel to the 
behavior parameter forming unit of each device. Said 
new behavior generation unit is programmed to gener- 
ate new patterns of behavior randomly or under prede- 
termined rules and to output the new patterns to the 
behavior unit of each device 

Other Applications 

[0161] The object to be controlled in the present 
invention is not limited. Any objects which interact with a 
user and which are allowed to output not only compul- 
sory (commanded) behavior but also autonomous 
behavior for the benefit of the user, as in the first and 
second embodiments, such as an engine in a motorbike 
or a car, an outboard engine in a vessel, an industrial 
robot used in machine tools, a communication partner 
robot a musical instrument a motor used in electrically- 
driven vehicles, a musical instrument such as a key- 
board, a research assistance system, home appliances 
such as an oven, or the like can be controlled by adopt- 
ing the control system of the present invention based on 
the same principle as in the aforesaid embodiments. 
Incidentally, the external information or environment 
information can be omitted if the object is used under 
limited circumstances or if the external or environment 
information has nothing to do with accomplishment of 
the user's command or if the user so desires. 
[0162] When the present invention is adapted to a 
control object in a different field or for a different pur- 
pose, by modifying items included in information to be 
processed, the control object can be controlled as with 
the aforesaid embodiments- In any case, according to 
the present invention, the control object, to which the 
present invention is adapted, is likely to be equipped 
with quasi-intelligence. 

[0163] tt will be understood by those of skill in the art 
that numerous variations and modifications can be 
made without departing from the spirit of the present 
invention. Therefore, it should be clearly understood 
that the forms of the present invention are illustrative 
only and are not intended to limit the scope of the 
present invention. 

Claims 

1. An autonomic device behaving adaptively to a user, 
comprising: 

(i) a recognition unit programmed to recognize 
a user-related status: 

(ii) a personality-forming unit programmed to 
form personality composed of preselected per- 
sonality elements when receiving signals of the 
user-related status accumulated for a predeter- 
mined period of time, based on a predeter- 
mined relationship between the preselected 
personality elements and the user-related sta- 
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tus; 

(iii) an emotion-forming unit programmed to 
form emotions composed of preselected emo- 
tion elements when receiving signals of the 
user-related status and the personality, based s 
on a predetermined relationship between the 
preselected emotion elements, the user- 
related status, and the personality; 
fiv) an autonomic behavior-establishing unit 
programmed to select a pattern of autonomic 10 
behavior composed of preselected elements 
when receiving signals of the user-related sta- 
tus, the personality and the emotions, based on 
a predetermined relationship between patterns 
of behavior, the user-related status, the per- is 
sonalrty, and the emotions; and 
(v) a new behavior pattern generation unit pro- 
grammed to generate new patterns of behavior 
by selecting and combining patterns' elements 
randomly or under predetermined rules, 20 
wherein the new patterns override the previous 
pattern under predetermined conditions. 

2. The device according to claim 1 , wherein the recog- 
nition unit further recognizes the user's command 25 
which is not included in the user-related status, the 
device further comprises: 

a commanded behavior-establishing unit pro- 
grammed to select a pattern of commanded 30 
behavior when receiving signals of the user's 
command; and 

a behavior control unit programmed to control 
behavior by combining the selected pattern of 
autonomic behavior and the selected pattern of 35 
commanded behavior, based on predeter- 
mined priority order. 6. 

3. The device according to claim 1 or 2, further com- 
prising: 40 

a sensing unit which senses signals from the 
user; 

a first memory which stores and updates the 7. 

recognized user-related status and/or the rec- 45 

ognized command; and 

a second memory which stores and updates 

the formed personality, wherein updating takes 

place under predetermined conditions. 

50 

4. The device according to any one of claims 1 to3, 
wherein the new behavior pattern generation unit 
has a database storing a pool of elements of pat- 
terns of autonomous behavior. 

55 

5. The device according to claim 3 or 4, wherein: 8. 

the sensing unit further senses predetermined 



user-unrelated signals relevant to accomplish- 
ment of the command but not part of the com- 
mand; 

the recognition unit further recognizes a cur- 
rent user-unrelated status based on the sensed 
predetermined signals; 

the first memory further stores and instantane- 
ously updates the recognized user-unrelated 
status; 

the pseudo-emotion-forming unit establishes 
each element of predetermined pseudo-emo- 
tion elements based further on the user-unre- 
lated status in addition to the user-related 
status stored in the first memory and the 
pseudo-personality stored in the second mem- 
ory to form pseudo-emotions; 
the predetermined relationship stored in the 
memory of the pseudo-emotion forming unit 
further includes the user-unrelated status to 
define a relationship between the predeter- 
mined pseudo-emotion elements, the user- 
related status, the user-unrelated status, and 
the pseudo-personality; 
the autonomic behavior-establishing unit 
selects a pattern of autonomic behavior based 
further on the user-unrelated status in addition 
to the user-related status, the pseudo-person- 
ality, and the pseudo-emotions; and 
the predetermined relationship stored in the 
memory of the autonomic behavior-forming 
unit further includes the user-unrelated status 
to define a relationship between patterns of 
autonomic behavior, the user-related status, 
the user-unrelated status, the pseudo-person- 
ality, and the pseudo-emotions. 

The device according to one of claims 1 to 5, 
wherein the new behavior pattern generation unit is 
linked to the pseudo-personality forming unit, 
wherein the new behavior pattern generation unit is 
activated at intervals correlated with the formed 
pseudo-personality. 

The device according to one of claims 3 to 6, further 
comprising a third memory which stores and 
updates the pseudo-emotion formed by the 
pseudo-emotion-forming unit, said third memory 
being linked to the behavior control unit wherein 
the second memory is further linked to the behavior 
control unit, wherein the pseudo-emotion stored in 
the third memory and the pseudo-personality 
stored in the second memory are used as parame- 
ters of the predetermined priority order provided in 
the behavior control unit 

The device accorcfing to one of claims 3 to 7, 
wherein the first memory is further linked to the 
behavior control unit, wherein the user-related sta- 
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tus stored in the first memory, the pseudo-personal- 
ity stored in the second memory, and the pseudo- 
emotion stored in the third memory are used as 
parameters of the predetermined priority order pro- 
vided in the behavior control unit 5 

9. A method for adapting behavior of a device to a 
user's characteristics, comprising the steps of: 

(b) recognizing user-related status; io 
(d) establishing personality composed of 
preselected personality elements based on 
received user-related status accumulated for a 
predetermined period of time based on a pre- 
determined relationship between the prese- is 
lected personality elements and the user- 
related status; 

(f) establishing emotions composed of prese- 
lected emotion elements based on received 
user-related status and the personality based 20 
on a predetermined relationship between the 
preselected emotion elements, the user- 
related status and the personality; 

(g) selecting a pattern of autonomic behavior 
conposed of preselected elements based on 2s 
received user-related status, the personality, 
and the emotions based on a predetermined 
relationship between patterns of autonomic 
behavior, the user-related status, the personal- 
ity, and the emotions; and 30 

(h) generating new patterns of autonomous 
behavior by selecting and combining patterns' 
elements randomly or under predetermined 
rules, wherein the new patterns override the 
previous pattern under predetermined condi- 35 
tions. 

1 0. The method according to claim 9, wherein further in 
step (b) the user's command is recognized which is 
not included in the user-related status; further com- 40 
prising the steps: 



wherein updating takes place under predeter- 
mined conditions. 

12. The method accord ng to one of clams 9 to 11, 
wherein in step (h) a pool of elements of patterns of 
autonomous behavior is stored with the use of a 
database. 

13. The method according to claim 11 or 12, further 
comprising: 

in step (a), sensing predetermined user-unre- 
lated signals relevant to accomplishment of the 
command but not part of the command; 
in step (b), recognizing a current user-unre- 
lated status based on the sensed predeter- 
mined signals; 

in step (c), storing and updating the recognized 
user-unrelated status; 

in step (d), establishing each element of prede- 
termined pseudo-emotion elements based fur- 
ther on the user-unrelated status in addition to 
the user-related status stored in step (c) and 
the pseudo-personality stored in step (e) to 
form pseudo-emotions, wherein the predeter- 
mined relationship stored in the memory fur- 
ther includes the user-unrelated status to 
define a relationship between the predeter- 
mined pseudo-emotion elements, the user- 
related status, the user-unrelated status, and 
the pseudo-personality; 
in step (g), selecting a pattern of autonomic 
behavior based further on the user-unrelated 
status in addition to the user-related status, the 
pseudo-personality, and the pseudo-emotions, 
wherein the predetermined relationship stored 
in the memory further includes the user-unre- 
lated status to define a relationship between 
patterns of autonomic behavior, the user- 
related status, the user-unrelated status, the 
pseudo-personality, and the pseudo-emotions. 



Q) selecting a pattern of commanded behavior 14, The method according to one of claims 9 to 13, 

in accordance with the received recognized wherein step (h) is activated at intervals correlated 

user's command; and 45 with the formed pseudo-personality, 
(k) controlling behavior by combining the 

selected pattern of autonomic behavior and the 1 & The method according to one of claims 9 to 14, fur- 
selected pattern of commanded behavior ther comprising the steps of storing and updating 
based on a predetermined priority order. the pseudo-emotion formed in step (f), wherein the 

so stored and updated pseudo-emotion and the 

11. The method according to cfaim 9 or~tOY further pseudo-personality stored in step (e) are used as 

comprising the steps of: parameters of the predetermined priority order pro- 
vided in step (k). 

(a) sensing signals from the user; 

(c) storing and updating the recognized user- ss 16. The method according to claim 15, wherein the 

related status and/or the recognized command; user-related status stored in step (c), the pseudo- 

and personality stored in step (e), and the stored and 

(e) storing and updating the formed personality, updated pseudo-emotion are used as parameters 
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of the predetermined priority order in step (k). 

17. A behavior control system comprising: 

a behavior unit programmed to behave based s 
on behavior parameters; 
an input unit programmed to output signals 
when receiving signals resulting from the 
behavior; and 

a memory medium attachable between the 10 
behavior unit and the input unit and detachable 
therefrom, said memory medium storing a rela- 
tionship between signals inputted from the 
input unit and behavior parameters outputted 
tot he behavior unit, said relationship pro- is 
grammed to be updated based on the signals 
from the input unit 

18. The behavior control system according to claim 17, 
wherein said memory medium is included in a per- 20 
sociality -forming unit which is programmed to deter- 
mine a relationship between signals inputted from 
the input unit and behavior parameters outputted to 
the behavior unit and which is programmed to 
update the relationship based on the signals from 2s 
the input unit stored in the memory medium. 

19. The behavior control system according to claim 17 
or 18, further comprising an emotion-forming unit 
provided between the input unit and the behavior 30 
unit in parallel to the personality-forming unit said 
emotion-forming unit programmed to determine a 
relationship between signals inputted from the input 
unit and second behavior parameters added to the 
first behavior parameters and outputted to the 35 
behavior unit based on the signals from the input 
unit. 

20. The behavior control system according to one of 
claims 17 to 19. further comprising a new behavior 40 
generation unit provided between the input unit and 
the behavior unit in parallel to the memory medium, 
said new behavior generation unit programmed to 
generate new patterns of behavior randomly or 
under predetermined rules and to output the new as 
patterns to the behavior unit 

21. A network of a plurality of autonomic devices, each 
behaving adaptively to a user and comprising: 

50 

a sensing unrf whIU t screes preselected sig- 
nals from a user who uses the autonomic 
device; 

a behavior unit programmed to behave based 
on behavior parameters; ss 
an input unit programmed to output signals 
when receiving signals resulting from the 
behavior through the sensing unit and signals 



provided through the network; and 
a behavior parameter-forming unit provided 
between the behavior unit and the input unit, 
said behavior parameter-forming unit compris- 
ing a memory storing a relationship between 
signals inputted form the input unit and behav- 
ior parameters outputted to the behavior unit, 
said behavior parameter-forming unit pro- 
grammed to update said relationship based on 
the signals from the input unit, wherein each 
device is connected with each other devices 
through the network. 

22. The network according to claim 21, wherein each 
device further comprises an emotion-forming unit 
provided between the input unit and the behavior 
unit in parallel to the behavior parameter-forming 
unit said emotion-forming unit programmed to 
determine a relationship between signals inputted 
from the input unit and second behavior parameters 
added to the first behavior parameters and output- 
ted to the behavior unit based on the signals from 
the input unit 

23. The network according to claim 21 or 22, wherein 
each device further comprises a new behavior gen- 
eration unit provided between the input unit and the 
behavior unit in parallel to the behavior parameter- 
forming unit said new behavior generation unit pro- 
grammed to generate new patterns of behavior ran- 
domly or under predetermined rules and to output 
the new patterns to the behavior unit. 

24w The network of a plurality of autonomic devices, 
each behaving adaptively to a user and comprising: 

a sensing unit which senses preselected sig- 
nals from a user who uses the autonomic 
device; 

a behavior unit programmed to behave based 
on behavior parameters; and 
an input unit programmed to output signals 
when receiving signals resulting from the 
behavior through the sensing unit and signals 
provided through the network; 
said network further comprising a main compu- 
ter connecting each device through the net- 
work, said computer comprising a behavior 
parameter-forming unit provided between the 
behavior unit and the input unit of each device 
ffrruugfrflte network, said behavior parameter- 
forming unit comprising a memory storing a 
relationship between signals inputted from the 
input unit and behavior parameters outputted 
to the behavior unit of each device, said behav- 
ior parameter-forming unit programmed to 
update said relationship of each device based 
on the signals from the input unit, said compu- 
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ter receiving the signals from the sensing unit 
or the input unit 

25. The network according to claim 24, wherein said 
computer further comprises an emotion-forming s 
unit provided between the input unit and the behav- 
ior unit in parallel to the behavior parameter-forming 
unit of each device through the network, said emo- 
tion-forming unit programmed to determine a rela- 
tionship between signals inputted from the input 10 
unit and second behavior parameters added to the 
first behavior parameters and outputted to the 
behavior unit of each device, based on the signals 
from the input unit of each device. 

15 

26. The network according to claim 24 or 25, wherein 
said computer further comprises a new behavior 
generation unit provided between the input unit and 
the behavior unit in parallel to the behavior parame- 
ter-forming unit of each device, said new behavior 20 
generation unit programmed to generate new pat- 
terns of behavior randomly or under predetermined 
rules and to output the new patterns to the behavior 
unit of each device. 
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